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Executive  Summary 


This  simulation  tested  the  procedure  for  simultaneous  Instrument  Landing  System  (ILS) 
approaches  to  two  parallel  runways  spaced  3000  ft  apart  with  one  localizer  offset  by  2.5  degrees. 
Controllers  monitored  traffic  using  the  Precision  Runway  Monitor  (PRM)  system,  which 
included  Final  Monitor  Aid  (FMA)  displays  and  a  simulated  radar  sensor  update  rate  of  1.0 
second. 

Aircraft  blunders  were  used  to  test  the  ability  of  the  system  to  maintain  distances  of  at  least  500  ft 
between  aircraft  during  critical  blunder  situations.  A  blunder  occurred  when  a  Target  Generation 
Facility  (TGF)  aircraft,  established  on  an  ELS  approach,  made  an  unexpected  30-degree  turn 
toward  an  aircraft,  usually  a  flight  simulator,  on  an  adjacent  approach.  Pilots  of  80  percent  of  the 
blundering  aircraft  were  instructed  to  disregard  controller  communications,  simulating  an 
inability  to  comply  with  controller  instructions.  Statistical  analyses  were  conducted  on  the 
nonresponding  aircraft  blunders  towards  flight  simulator  targets.  Test  criterion  violations 
(TCVs)  occurred  when  the  separation  between  aircraft  was  less  than  500  ft.  Blunders  that  would 
have  resulted  in  aircraft  miss  distances  of  less  than  500  ft  had  there  been  no  controller 
intervention  were  classified  as  at-risk. 

The  following  criteria  were  developed  by  the  Multiple  Parallel  Approach  Program  (MPAP) 
Technical  Work  Group  (TWG)  to  evaluate  the  study: 

•  the  number  of  TCVs  relative  to  the  total  number  of  at-risk,  nonresponding  blunders  and  to  a 
predetermined  target  level  of  safety  of  no  more  than  one  fatal  accident  per  25,000,000 
approaches; 

0  Both  the  real-time  simulation  data  and  Monte  Carlo  techniques  were  used  for  this 
assessment. 

•  the  frequency  of  No  Transgression  Zone  (NTZ)  entries  and  nuisance  breakouts  (NBOs); 

0  NTZ  entries  occurred  when  aircraft  entered  the  NTZ,  not  including  aircraft  that  were 
directed  to  blunder.  NBOs  resulted  when  aircraft  were  broken  out  of  the  final  approach 
for  reasons  other  than  a  blunder,  loss  of  longitudinal  separation,  or  lost  beacon  signal. 

•  an  evaluation  of  controller  communications  workload;  and 

•  an  operational  assessment  from  MPAP  TWG  members,  based  on  their  expertise  and 
judgment,  and  on  evaluations  from  participating  controller  technical  observers  and  pilot 
technical  observers. 

During  the  simulation,  284  blunders  were  initiated  that  would  have  resulted  in  aircraft  miss 
distances  of  less  than  500  ft  had  the  controllers  not  intervened.  Of  the  284  blunders,  253  were 
nonresponding,  making  them  worst-case  blunders  (WCBs).  No  TCVs  occurred  in  any  of  the 
WCB  situations.  The  confidence  interval  for  the  true  TCV  rate,  based  on  the  real-time  results, 
was  0.0  to  2.1  percent.  The  TCV  rates  resulting  from  the  two  Monte  Carlo  simulations  were  1.9 
±  0.1  percent  and  0.1 1  ±  0.01  percent.  These  results  were  both  consistent  with  the  real-time 
simulation  results  and  below  the  test  criterion  of  6.8  percent.  In  addition,  the  target  level  of 
safety  of  no  more  than  one  fatal  accident  per  25  million  approaches  was  achieved. 
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An  evaluation  of  NTZ  entries  and  NBOs  was  conducted  to  assess  system  capacity  and  controller 
workload.  In  the  approach  course  configuration,  no  NTZ  entries  occurred  as  a  result  of  Total 
Navigation  System  Error  (TNSE).  TNSE  was  defined  for  this  simulation  as  the  difference 
between  the  actual  flight  path  of  the  aircraft  and  its  intended  flight  path.  NBOs  occurred  in  1.3 
percent  of  all  nonblunder-related  approaches  due  to  TNSE-related  events  (i.e.,  aircraft 
approaching  the  NTZ).  Both  results  were  considered  acceptable. 

The  TWG  determined  that  the  controller  communications  workload  associated  with  TNSE- 
related  events  was  at  a  satisfactory  level  based  on  their  observations  during  the  simulation  and  on 
questionnaires  from  participating  controllers. 

The  MPAP  TWG  unanimously  agreed  that  this  3000-ft  configuration  met  all  of  the  test  criteria. 
Participating  controller  and  pilot  technical  observers  also  supported  this  position.  The 
development  of  pilot  training  and  modifications  to  the  controller  training  were  significant  in 
enabling  a  successful  operation. 

The  pilot  training  consisted  of  viewing  a  video,  RDU  Precision  Runway  Monitor:  A  Pilot's 
Approach  (FAA,  1995c),  reading  a  Pilot  Awareness  Training  Bulletin,  and  taking  a  self- 
administered  test  on  pilot  awareness.  The  pilots  who  flew  the  glass  cockpit  simulators,  the 
MD90  and  B747-400,  were  also  required  to  read  a  Breakout  Procedure  Bulletin  and  complete  a 
self-administered  test  on  the  subject.  All  pilots  were  required  to  hand-fly  any  ATC-directed 
breakout,  which  significantly  improved  the  pilot/aircraft  breakout  performance. 

Controllers  were  trained  extensively  for  this  simulation.  They  were  each  given  a  mandatory  8 
hours  of  hands-on  training.  The  training  period  was  used  for  familiarization  with  the  PRM 
equipment.  Controllers  were  able  to  observe  and  attempt  to  resolve  blunders  while  using  new 
breakout  phraseology  that  was  developed  as  a  result  of  information  gained  from  previous 
simulations.  In  addition  to  the  hands-on  training,  controllers  were  educated  on  procedures  used 
in  the  cockpit  after  breakout  instructions  were  received.  This  was  accomplished  through  video 
recordings  of  simulator  cockpits.  The  purpose  of  the  video  presentation  was  to  allow  controllers 
to  make  educated  decisions. 

In  addition  to  the  training  developments  and  modifications,  the  TWG  agreed  that,  in 
implementing  the  close  parallel  configuration,  anti-blocking  devices  (ABDs)  will  be  required  in 
the  ground  radios.  In  the  event  that  ABDs  are  not  available,  the  air  traffic  facility  will  have  two 
communication  frequencies  available  for  each  close  parallel  runway,  allowing  the  tower  and 
monitor  controllers  to  simultaneously  broadcast  on  both  frequencies  to  reduce  the  effect  of 
blocked  communications. 

The  test  results  demonstrated  that  the  addition  of  controller  and  pilot  training  improved  the 
procedure.  Controller  and  pilot  responsibilities  were  clearly  understood,  response  times  were 
sufficient,  and  adequate  aircraft  separation  was  maintained  at  all  times.  The  TWG,  therefore, 
recommended  the  procedure  on  simultaneous  approaches  to  dual  runways  spaced  3000  ft  apart 
with  one  localizer  offset  by  2.5  degrees  for  approval  in  the  operational  environment,  given 
similar  controller  and  pilot  training,  when  the  PRM  system  with  a  1.0-second  update  rate  is  used. 
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1.  Introduction 


The  ability  of  the  National  Airspace  System  (NAS)  to  meet  future  air  traffic  demands  is  a  serious 
concern.  Programs  to  improve  NAS  capacity  have  been  underway  since  the  early  1980s,  both  to 
reduce  air  traffic  delays  and  to  accommodate  the  increased  demand.  Contributing  to  capacity 
problems  are  the  limitations  imposed  by  current  airport  runway  configurations  and  the  associated 
air  traffic  separation  criteria,  particularly  as  related  to  aircraft  executing  instrument  landing 
system  (ILS)  approaches  under  instrument  meteorological  conditions  (IMC). 

One  way  to  improve  system  capacity  and  efficiency  is  to  reduce  the  minimum  runway  spacings 
for  parallel  approach  procedures.  This  approach  permits  current  airports  to  be  modified  rather 
than  requiring  that  new  airports  be  built.  Dual  simultaneous  ILS  approaches  are  authorized, 
using  current  radar  technology,  to  runways  spaced  4300  ft  and  greater.  In  addition,  dual 
simultaneous  independent  ILS  approaches  are  authorized  to  runways  spaced  3400  to  4300  ft  apart 
when  Precision  Runway  Monitor  (PRM)  systems  with  radar  updates  of  2.4  seconds  or  less  are 
used  (Federal  Aviation  Administration  (FAA)  1995a).  PRM  systems  provide  controllers  with 
high-precision  secondary  surveillance  data  and  more  frequent  aircraft  target  update  rates  for 
monitoring  aircraft  on  final  approaches.  PRM  systems  allow  simultaneous  ILS  approaches  to  be 
conducted  where  they  were  previously  restricted  due  to  existing  runway  spacings  and  radar  error. 

1 . 1  The  Multiple  Parallel  Approach  Program  Technical  Work  Group 

The  Multiple  Parallel  Approach  Program  (MPAP)  Technical  Work  Group  (TWG)  has  been 
investigating  the  use  of  triple,  quadruple,  and  closely  spaced  dual  parallel  runway  configurations 
through  the  conduct  of  real-time  simulations  at  the  William  J.  Hughes  Technical  Center  in 
Atlantic  City  International  Airport,  NJ  (Appendix  A).  The  MPAP  TWG  is  composed  of  FAA 
representatives  from  the  Secondary  Surveillance  Product,  Office  of  System  Capacity  and 
Requirements,  Flight  Standards  Service,  Air  Traffic  Rules  and  Procedures  Service,  Office  of  Air 
Traffic  System  Management,  and  Air  Traffic  Plans  and  Requirements  Service. 

The  MPAP  TWG  brings  together  various  areas  of  expertise  to  evaluate  the  feasibility  of  multiple 
parallel  approaches  in  an  effort  to  increase  airport  capacity  in  a  safe  and  acceptable  manner.  The 
TWG  evaluates  simulated  proposed  operations  against  specific  test  criteria  that  have  been 
developed  over  the  course  of  many  real-time  simulations  (see  Section  2).  Only  after  extensive 
review  and  evaluation  of  simulation  results  does  the  TWG  conclude  whether  or  not  a  proposed 
procedure  should  be  recommended  for  approval  in  the  operational  environment. 

1.2  Simulation-Related  Definitions 

The  TWG  has  developed  definitions  and  classifications  that  are  specific  to  the  MPAP  real-time 
simulations.  The  following  sections  explain  the  simulation-related  terms  that  are  referred  to 
throughout  the  report. 
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1.2.1  Blunders 


During  MPAP  simulations,  aircraft  blunders  are  initiated  to  measure  the  ability  of  the  system  to 
maintain  adequate  separation  between  aircraft  on  final  approaches  during  critical  situations.  A 
blunder  occurs  when  an  aircraft,  already  established  on  the  final  approach  course,  makes  an 
unexpected  turn  towards  another  aircraft  on  an  adjacent  approach  (see  Figure  1).  Adequate 
separation  is  maintained  and  the  blunder  is  considered  resolved  if  the  minimum  slant  distance 
between  the  blundering  aircraft  and  the  evading  aircraft  at  the  closest  proximity  is  500  ft  or 
greater. 
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Figure  1.  Aircraft  blunder  during  parallel  approach  operations. 

1 .2. 1 . 1  Test  Criterion  Violations 

Any  blunder  that  results  in  a  miss  distance  of  less  than  500  ft  between  aircraft  is  considered  a 
Test  Criterion  Violation  (TCV).  A  valid  TCV  is  one  that  could  occur  in  the  operational 
environment  for  any  number  of  reasons  and  is  not  the  result  of  a  simulation  anomaly  (e.g., 
simulation  hardware  and  software  failure).  If  a  blunder  results  in  a  TCV,  the  blunder  is 
considered  unresolved. 

1 .2. 1 .2  Blunder  Classifications 

One  way  to  classify  blunders  is  by  the  severity  of  the  situation.  For  instance,  blunders  are 
classified  as  at  risk  or  not  at  risk.  An  at-risk  blunder  is  one  that  would  have  resulted  in  a  miss 
distance  of  less  than  500  ft  had  evasive  maneuvers  not  been  executed  by  either  the  blundering  or 
the  evading  aircraft.  An  at-risk  blunder  is  determined  mathematically,  based  upon  the  projected 
courses  of  the  blundering  and  evading  aircraft  at  the  start  of  the  blunder.  A  blunder  that  is  not  at 
risk  is  one  that  would  have  resulted  in  a  miss  distance  of  500  ft  or  greater  without  any  evasive 
maneuvers  being  taken. 
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Blundering  aircraft  are  also  classified  as  responding  or  nonresponding.  A  responding  aircraft  is 
one  in  which  the  pilot  of  the  blundering  aircraft  verbally  responds  with  the  controller's  instructions 
and  attempts  to  return  the  aircraft  to  the  localizer  course  or  execute  some  other  evasive  maneuver. 
A  nonresponding  aircraft  is  one  in  which  the  test  director  instructs  the  pilot  to  disregard  controller 
communications,  simulating  an  inability  to  correct  the  deviation  from  the  approach  course.  This 
inability  to  correct  a  blunder  is  meant  to  simulate  situations  involving  communications  problems, 
hardware  failures,  or  human  error. 

Blundering  aircraft  are  scripted  to  turn  at  predetermined  angles  towards  adjacent  approach 
courses.  In  this  simulation,  all  blundering  aircraft  executed  30-degree  turns.  A  worst-case 
blunder  (WCB)  occurs  when  the  blundering  aircraft  turns  at  an  angle  of  30  degrees  and  is 
nonresponding.  In  addition,  blundering  aircraft  may  either  maintain  altitude  or  descend. 

1,2,2  No  Transgression  Zone  Entries  and  Nuisance  Breakouts 

The  final  approach  airspace  is  divided  into  two  areas  between  the  runways,  the  Normal  Operating 
Zone  (NOZ)  and  the  No  Transgression  Zone  (NTZ),  as  shown  in  Figure  2.  The  NOZ  is  the  area 
between  the  NTZ  and  the  final  approach  course  where  aircraft  are  permitted  to  fly.  The  NTZ  is 
the  2000-ft  wide  area  equidistant  between  final  approach  courses  where  aircraft  are  not  permitted 
to  enter. 
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Figure  2.  Normal  Operating  Zone  and  No  Transgression  Zone. 

If  an  aircraft  enters  the  NTZ,  FAA  regulations  require  the  monitor  controller  to  break  any 
adjacent  aircraft  out  of  the  approach.  Because  the  NTZ  is  fixed  at  2000  ft,  the  NOZ  varies  with 
runway  separation.  As  separation  between  runways  decreases,  the  NOZ  decreases,  providing  less 
airspace  for  aircraft  to  fly  along  the  ILS  and  a  greater  opportunity  for  aircraft  to  enter  the  NTZ. 
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As  runways  become  more  closely  spaced,  Total  Navigation  System  Error  (TNSE)  becomes  a 
concern.  TNSE  represents  the  difference  between  the  actual  flight  path  of  an  aircraft  and  its 
intended  flight  path.  TNSE  can  be  caused  by  flight  technical  error,  avionics  error,  ILS  signal 
error,  and/or  weather.  TNSE  may  contribute  to  the  occurrence  of  NTZ  entries  and  nuisance 
breakouts  (NBOs).  An  NTZ  entry  occurs  when  an  aircraft  enters  the  NTZ  for  reasons  other  than 
a  blunder  or  breakout.  An  NBO  occurs  when  an  aircraft  is  broken  out  of  its  final  approach 
course  for  reasons  other  than  a  blunder,  loss  of  longitudinal  separation,  or  lost  beacon  signal  (i.e., 
aircraft  target  goes  into  coast). 

2.  Acceptance  Criteria 

The  MPAP  TWG  uses  four  criteria  to  evaluate  simulated  operations:  the  TCV  rate  and  risk 
analysis,  the  frequency  of  NTZ  entries  and  NBOs,  an  evaluation  of  controller  communications 
workload,  and  a  TWG  operational  assessment. 

2.1  Test  Criterion  Violation  Rate  and  Risk  Analysis 

2.1.1  Test  Criterion  Violation  Rate  Derivation 

The  TCV  rate  is  a  measure  of  the  system  blunder  resolution  capability.  Individual  blunders  are 
evaluated  to  determine  whether  or  not  they  are  at-risk.  The  number  of  TCVs  divided  by  the 
number  of  at-risk  blunders  results  in  an  initial  estimate  of  the  TCV  rate.  The  number  of  at-risk, 
nonresponding  blunders  that  occur  during  the  real-time  simulation,  however,  is  relatively  low, 
and  therefore  a  large  confidence  interval  results. 

To  ensure  a  more  accurate  measurement  of  the  operational  TCV  rate,  this  criterion  is  measured 
using  two  fast-time  computer,  or  Monte  Carlo,  simulations.  The  Monte  Carlo  simulations  (i.e., 
the  Blunder  Risk  Model  (BRM)  and  the  Airspace  Simulation  and  Analysis  for  Terminal 
Instrument  Procedures  (TERPS)  (ASAT)  Model)  each  use  data  collected  in  the  real-time 
simulation  to  model  over  100  thousand  at-risk  blunders  each,  thus  reducing  the  range  of  the 
confidence  interval  to  a  very  small  size.  The  methods  used  in  the  Monte  Carlo  simulations  are 
described  in  Appendix  B. 

The  TCV  rate  estimates  from  the  Monte  Carlo  simulations  are  compared  to  the  results  of  the  real¬ 
time  simulation  to  ensure  that  they  are  consistent.  Specific  procedures  for  the  evaluation  of  TCV 
rate  and  risk  analysis  are  contained  in  Section  1 1.2  and  in  Appendix  C. 

2.1.2  Maximum  Acceptable  Test  Criterion  Violation  Rate 

In  the  PRM  Demonstration  Report  (1991),  a  TCV  rate  was  computed  from  the  population  of  all 
WCBs.  It  was  found  that  a  TCV  rate  not  greater  than  0.004  TCV  per  WCB  would  meet  the 
target  level  of  safety,  provided  that  the  overall  30-degree  blunder  rate  did  not  exceed  one  30- 
degree  blunder  per  2000  approaches.  The  real-time  simulation,  however,  measures  a  TCV  rate 
based  on  at-risk  WCBs,  not  the  population  of  all  WCBs.  Therefore,  for  comparison  purposes, 
the  population  TCV  rate  is  converted  to  an  at-risk  TCV  rate.  Based  on  a  simulation  of  aircraft 
speeds  and  types,  a  conservative  ratio  of  1/17  at-risk  WCB  per  WCB  is  applied,  resulting  in  an 
at-risk  TCV  rate  criterion  of  6.8  percent  for  this  simulation  (see  Appendix  C). 
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2. 1 .3  Relationship  Between  Test  Criterion  Violation  Rate  and  Risk  Analysis 

For  this  3000-ft,  2.5-degree  localizer  offset  simulation,  a  Monte  Carlo  at-risk  TCV  rate 
confidence  interval  not  exceeding  6.8  percent  would  indicate  a  fatal  accident  rate  below  the 
target  level  of  safety  and  would  be  acceptable.  A  Monte  Carlo  confidence  interval  that  extends 
above  6.8  percent  would  indicate  that  the  operation  does  not  meet  the  target  level  of  safety. 

2.2  Frequency  of  No  Transgression  Zone  Entries  and  Nuisance  Breakouts 

Measuring  the  frequency  of  NTZ  entries  and  NBOs  provides  an  assessment  of  how  TNSE  may 
have  affected  the  simulated  approach  configuration.  All  NTZ  entries  and  NBOs  that  occur  as  a 
result  of  TNSE  are  examined.  The  frequency  of  NTZ  entries  and  NBOs  has  to  be  at  an 
acceptable  level  as  determined  by  the  MPAP  TWG. 

2.3  Controller  Communications  Workload 

The  controller  communications  workload  criterion  has  been  developed  as  a  result  of  past 
simulation  observations  of  the  effects  of  TNSE-related  events.  As  runways  become  more  closely 
spaced,  the  opportunity  for  NTZ  entries  and  NBOs  increases,  as  does  radio  frequency  congestion 
due  to  those  TNSE-related  events.  The  TWG  makes  a  subjective  evaluation  of  the  acceptability 
of  the  communications  workload  required  of  the  controllers  to  maintain  aircraft  flight  courses 
within  the  NOZ. 

2.4  Technical  Work  Group  Operational  Assessment 

An  operational  assessment  of  the  approach  configuration  is  conducted  by  MPAP  TWG  members. 
The  assessment  reflects  the  TWG’s  overall  evaluation  of  the  simulated  procedure  and 
recommendation  regarding  the  feasibility  of  implementing  the  procedure  in  the  operational 
environment.  The  operational  assessment  is  based  on  all  test  results,  on  MPAP  TWG  expertise 
and  judgment,  and  on  evaluations  from  subject  controllers,  participating  controller  technical 
observers,  and  pilot  technical  observers. 

3.  Simulation  Overview 

3. 1  Previous  3000-ft  Simulation 

In  July  1994,  a  simulation  was  conducted  to  evaluate  dual  parallel  runways  spaced  3000  ft  apart 
with  one  localizer  offset  by  2.5  degrees  using  the  PRM  system.  Although  some  of  the  test 
acceptance  criteria  were  met,  the  TWG  did  not  recommend  the  simulated  procedure,  as  tested. 
The  following  three  specific  problems  areas  were  identified: 

a.  pilot  training  (i.e.,  pilot/aircraft  response  times  following  breakout  instructions), 

b.  communications  (i.e.,  blocked  or  clipped  breakout  instructions),  and 

c.  controller  training  (i.e.,  controller  breakout  phraseology  and  procedures). 
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3.1.1  Pilot  Training 


Aircraft  breakout  maneuvering  responses  were  inadequate  to  meet  the  target  level  of  safety 
defined  in  the  test  criteria  (i.e.,  1  fatal  accident  in  25  million  approaches).  Pilots  were  not  trained 
to  perform  the  maneuver  and  were  unfamiliar  with  Air  Traffic  Control  (ATC)-directed  breakouts 
between  glide  slope  intercept  and  decision  height.  Unfamiliarity  with  breakouts  was 
compounded  by  specific  procedures  required  by  highly  automated  flight  control  systems  (i.e., 
flight  director,  flight  management  system,  autopilot).  The  TWG,  therefore,  decided  to  determine 
if  formal  training  would  improve  situational  awareness  and  operational  performance. 

3.1.2  Communications 

Blocked  and  clipped  communications  contributed  to  the  severity  of  many  blunders  throughout 
the  simulation.  The  time  available  for  breakout  communications  was  limited  and  did  not  allow 
for  identification  of  blocked  communications  and  subsequent  rebroadcasts.  When 
communications  were  blocked  or  clipped,  aircraft  responding  to  the  repeated  transmissions  were 
frequently  not  able  to  execute  evasive  maneuvers  in  a  timely  manner.  The  TWG  proposed 
investigating  methods  to  identify  and  resolve  blocked  and  clipped  communications. 

3.1.3  Controller  Training 

Prior  to  the  simulation,  controllers  were  briefed  on  the  standard  phraseology  to  be  used  in  the 
event  of  a  blunder.  During  the  simulation,  however,  breakout  phraseologies  varied  in  content 
and  duration.  This  resulted  in  two  effects.  First,  pilots  did  not  always  receive  both  the  heading 
and  altitude  assignment  in  the  initial  breakout  transmissions,  causing  some  delays.  Second, 
nonstandard  phraseology  usage  sometimes  resulted  in  a  lack  of  urgency  or  confusion  to  the  flight 
crews.  Controller  transmissions  may  have  been  less  ambiguous  had  more  emphasis  been  placed 
on  the  use  of  standardized  phraseology.  The  TWG  proposed  investigating  methods  to  improve 
controller  training. 

3.2  Multiple  Parallel  Approach  Program  Investigative  Groups 

Over  the  course  of  a  year,  action  was  taken  to  resolve  the  problem  areas  identified  in  the  July 
1994  simulation.  TWG  members,  controller  technical  observers,  and  pilot  technical  observers 
participated  in  MPAP-formed  subgroups  or  investigative  groups,  whose  goals  were  to  explore 
ways  to  improve  performance  in  each  of  the  problem  areas  through  additional  studies.  The  TWG 
believed  that  such  improvements  could  effect  a  successful  3000-ft,  2.5-degree  offset  operation. 

3.2. 1  Pilot  Training  Investigative  Group 

The  MPAP  TWG  conducted  two  studies  to  determine  the  effect  of  training  on  pilot  breakout 
performance.  Since  the  time  to  initiate  the  breakout  maneuver  was  longer  in  glass  cockpit 
aircraft  than  in  older  analog-type  aircraft,  the  studies  were  conducted  in  two  glass  cockpit 
simulators,  a  B747-400  and  an  A320.  Two  areas  of  training  were  identified:  pilot  awareness 
training  and  aircraft-specific  breakout  procedure  training. 
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The  studies  were  divided  into  several  phases,  with  each  phase  containing  more  training  than  the 
preceding  phase.  The  first  phase  used  pilots  without  training  to  establish  a  baseline  for  breakout 
performance.  The  next  phase  determined  the  effect  of  pilot  awareness  training  on  breakout 
performance.  Pilot  awareness  training  consisted  of  viewing  a  video  (FAA,  1995c)  about  the 
PRM  system,  reading  a  Pilot  Awareness  Bulletin,  and  taking  a  short  self-administered  test. 
Although  the  pilot  awareness  training  improved  the  breakout  performance,  it  was  felt  that  more 
improvement  was  needed.  The  next  phase  added  aircraft-specific  breakout  procedure  training. 
This  training  consisted  of  the  pilot  reading  a  Procedure  Bulletin  and  completing  a  short  self- 
administered  test. 

It  was  observed  during  these  simulations  that  the  hand-flown  breakouts  were  performed  more 
quickly  than  the  autopilot-flown  breakouts.  This  observation  led  to  the  next  phase  of  testing,  a 
comparison  of  hand-flown  breakout  performance  and  autopilot-flown  breakout  performance. 

The  B747-400  was  selected  for  this  study  because  its  autopilot  breakout  performance  was  slower 
than  that  of  the  A320.  There  was  a  significant  improvement  in  breakout  performance  when  the 
B747-400  was  hand-flown  during  a  breakout  as  compared  to  when  the  autopilot  was  used  to 
perform  the  maneuver. 

The  conclusions  reached  during  this  series  of  mini-simulations  significantly  influenced  the  pilot 
training  program  used  in  the  October  1995  3000-ft  simulation. 

3.2.2  Communications  Investigative  Group 

The  occurrence  of  blocked  communications  was  identified  as  a  major  factor  contributing  to  the 
severity  of  blunder  outcomes  in  the  July  1994  simulation.  The  controllers  were  often  unaware  of 
the  blocked  communications,  resulting  in  longer  system  response  times.  To  combat  this 
problem,  the  communications  investigative  group  began  research  on  VHF/UHF  anti-blocking 
devices  (ABDs).  These  ABDs  would  provide  the  controller  and/or  aircrew  with  an  indication 
that  outgoing  messages  were  blocked  by  someone  else's  transmission.  An  ABD  has  been 
developed  and  implemented  in  England.  Several  of  the  devices  were  acquired  and  are  currently 
being  tested  for  compatibility  with  FAA  approved  systems.  Since  the  ABD  testing  was  not 
complete  at  the  time,  ABDs  were  not  incorporated  into  the  October  1995  simulation.  However, 
the  communications  issue  was  addressed  indirectly  in  the  October  simulation  through  the 
improved  controller  training  package,  which  included  new  breakout  phraseology. 

3.2.3  Controller  Training  Investigative  Group 

The  controller  training  investigative  group  made  several  modifications  to  the  controller  training 
syllabus  in  an  effort  to  increase  controller  awareness  and  preparedness  for  monitoring  closely 
spaced  approach  configurations.  One  significant  change  involved  the  controller  breakout 
instruction  phraseology. 

In  the  July  1994  3000-ft  simulation,  the  controller  breakout  phraseology  from  FAA  Order 
71 10.65H  (FAA,  1995a)  was  used,  which  consisted  of  the  aircraft  call  sign  followed  by  the 
breakout  instructions.  Due  to  excessive  clipped  aircraft  call  signs  at  the  beginning  of  the 
messages,  pilots  did  not  always  know  who  the  calls  were  for  and,  thus,  did  not  always  respond. 
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Therefore,  changes  were  made  to  the  phraseology  and  tested  in  an  August  1995  triple  approach 
simulation.  The  modified  phraseology  began  with  the  aircraft  call  sign,  followed  by  the  addition 
of  the  phrase,  "Traffic  Alert",  which  was  used  in  an  effort  to  increase  awareness  of  the  urgency  of 
the  situation,  followed  by  a  repeat  of  the  aircraft  call  sign,  followed  by  the  instructions.  After  the 
simulation,  it  was  determined  that  the  phraseology  resolved  the  clipped  communications 
problems  (with  the  repeat  of  the  aircraft  call  sign),  but  the  phraseology  message  itself  was  too 
lengthy  and  delivery  was  poor  in  many  situations. 

As  a  result,  the  phraseology  was  modified  once  more  for  the  October  1995  3000-ft,  2.5-degree 
localizer  offset  simulation.  This  time  it  was  shortened  by  omitting  the  first  aircraft  call  sign  to 
allow  for  an  easier,  quicker  delivery  of  instructions.  Putting  the  phrase,  "Traffic  Alert",  at  the 
beginning  of  the  message  had  a  twofold  benefit:  heighten  the  awareness  of  all  listeners  on  the 
frequency  as  to  the  urgency  of  the  impending  situation  and  help  prevent  the  clip  of  the  aircraft 
call  sign  prior  to  the  breakout  instruction. 

In  addition  to  the  new  breakout  phraseology,  the  modified  controller  training  syllabus  also 
emphasized  the  need  for  a  timely  response  from  the  controller  and  highlighted  the  effect  of  the 
controller  breakout  instruction  on  the  aircrew's  workload.  A  video  was  developed  using  video 
recordings  from  previous  simulations,  which  demonstrated  reactions  and  responses  of  flight 
crews  to  breakout  instructions,  including  instructions  to  descend.  The  training  syllabus  stressed 
the  importance  of  completing  the  prescribed  phraseology  in  one  transmission.  This  was  based  on 
past  observations  that  information  in  a  later  transmission  was  sometimes  missed  due  to  breakout 
activity  in  the  cockpit  or  to  blocked  or  clipped  communications  as  the  result  of  frequency  usage. 

The  amount  of  controller  hands-on  training  was  changed  to  require  controllers  to  complete  a 
mandatory  8  hours  on  position  prior  to  participating  in  actual  test  runs.  This  allowed  for 
sufficient  practice  of  blunder  detection  and  the  use  of  the  prescribed  breakout  phraseology. 

3.3  October  1995  3000-ft  Simulation 

In  October  1995,  a  simulation  was  conducted  to  re-evaluate  the  procedures  for  simultaneous 
independent  ILS  approaches  to  two  parallel  runways  spaced  3000  ft  apart  with  one  localizer 
offset  by  2.5  degrees.  This  study  is  the  focus  of  the  remainder  of  this  report. 

3.3.1  Airport  Configuration 

Controllers  monitored  traffic  using  a  simulated  PRM  system  with  a  1.0-second  update  rate.  The 
airport  layout,  runways,  and  arrival  frequencies  emulated  an  airport  with  even  thresholds,  glide 
slopes  of  3  degrees,  and  field  elevation  of  500  ft  MSL  (see  Figure  3).  The  turn-on  altitude  for 
runway  18R  was  4500  ft  MSL  with  a  glide  slope  intercept  of  12.40  nm.  The  turn-on  altitude  for 
runway  18L  was  3500  ft  MSL  with  a  glide  slope  intercept  of  9.26  nm. 

3.3.2  Test  Runs 

The  simulation  was  conducted  over  a  2-week  period,  Saturdays  and  Sundays  excluded.  Three 
2-hour  runs  were  conducted  each  day.  A  total  of  10  runs  (5  runs  for  each  controller  group)  were 
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AIRPORT  DATA 

Field  Elevation: 

500  ft  msl 

Magnetic  Variation: 

0 

Runways: 

10,000  ft  x  150  ft 

Altimeter: 

29.92 

Weather: 

300  ft  ceiling  and  l 

3/4  mile  visibility 

Figure  3.  Airport  configuration  for  a  2.5-degree  localizer  offset. 
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dedicated  to  the  controller  briefing  and  training.  Also,  only  two  runs  were  conducted  on  the  last 
day  of  the  two  weeks  of  testing.  Therefore,  data  from  a  total  of  19  runs  were  collected  and 
analyzed. 

4.  Experimental  Apparatus 

The  development  of  the  real-time  simulation  environment  at  the  William  J.  Hughes  Technical 
Center  in  Atlantic  City,  NJ  has  made  real-time  simulation  testing  one  of  the  most  advanced 
methods  for  evaluating  ATC  procedures  development.  The  Technical  Center  laboratories 
contain  fully-operational  ATC  displays  that  have  the  capability  to  interface  with  remote  flight 
simulators  across  the  country.  With  this  end-to-end  simulation  capability,  tests  can  be  conducted 
to  collect  data  on  controller  and  pilot  performance  issues  that  cannot  be  measured  in  the 
operational  environment. 

4. 1  Target  Generation  Facility  Laboratory 

The  TGF  is  an  advanced  simulation  system  designed  to  support  testing  of  current  and  future  ATC 
systems  at  the  William  J.  Hughes  Technical  Center.  For  this  simulation,  the  functionality  of  the 
TGF  system  was  partitioned  into  three  subsystems:  simulation  pilot,  target  generation,  and 
development  and  support. 

The  simulation  pilot  workstations  (SPWs)  were  computer  workstations  containing  a 
communication  system  that  provided  an  audio  interface  with  the  monitor  controllers.  Simulation 
pilot  operators  (SPOs)  used  the  SPWs  to  fly  the  simulated  aircraft  and  commanded  them  in 
accordance  with  ATC  instructions. 

The  target  generation  (TG)  subsystem  consisted  of  a  TG  chassis  and  an  external  interface  (El) 
chassis.  The  TG  performed  all  modeling  within  the  TGF  and  correlated  dynamic  data,  such  as 
aircraft  state  vectors  and  radar  performance,  with  known  flight  plans.  The  El  was  responsible  for 
creating  the  exact  form  and  content  of  the  digitized  radar  messages  sent  to  the  ATC  system  under 
test.  Controller-pilot  voice  communications  were  processed  through  an  AMECOM  voice 
communications  system. 

The  development  and  support  subsystem  provided  the  basic  post-exercise  data  reduction  and 
analysis  capabilities.  In  addition,  this  subsystem  provided  the  capabilities  necessary  to  maintain 
and/or  enhance  the  TGF  software. 

In  total,  the  TGF  modeled  a  logical  view  of  the  ATC  environment,  including  long  and  short  range 
radar  sensors,  controlled  airspace,  weather  conditions,  air  traffic,  and  aircraft  performance.  The 
TGF  configuration  for  the  3000-ft,  2.5-degree  offset  simulation  is  shown  in  Figure  4. 

4.2  Radar  System  and  Controller  Displays 

The  final  monitor  controllers  used  prototypes  of  the  components  of  the  PRM  system  located  in  the 
Systems  Display  Laboratory  at  the  William  J.  Hughes  Technical  Center.  The  components 
consisted  of  Final  Monitor  Aid  displays  and  a  simulated  electronic  scanning  beacon  sensor  with  a 
1.0-second  update  rate. 
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Figure  4.  Target  generation  facility  configuration. 

4.2. 1  Final  Monitor  Aid  Displays 

FMAs  are  high  resolution  color  displays  equipped  with  the  controller  alert  system  hardware  and 
software  used  in  the  simulated  PRM  system.  The  FMA  displays  used  in  this  simulation  included 
alert  algorithms  that  provided  the  target  predictors,  color  change  alerts  when  targets  penetrated  or 
were  predicted  to  penetrate  the  NTZ,  color  change  alerts  if  the  aircraft  transponders  became 
inoperative,  synthesized  voice  alerts,  digital  mapping,  and  other  features  contained  in  the  PRM 
system. 

Graphics  for  the  FMAs  used  during  the  simulation  were  generated  by  a  Metheus  graphics  driver, 
and  the  operating  system  was  driven  by  a  micro-VAX  computer.  In  addition  to  the  mapping 
information  currently  provided  by  ARTS  displays,  the  FMAs  provided  features  to  aid  controllers 
in  the  early  detection  of  blunders  and  the  control  of  airspace.  These  included  independent 
magnification  capabilities,  color  coding,  aircraft  predictor  lines,  and  audio  and  visual  warnings. 

FMAs  provide  the  capability  to  adjust  the  horizontal  and/or  vertical  ratio  of  the  display. 
Horizontal  and  vertical  (X-Y)  axes  can  be  scaled  independently  to  improve  the  controller's  ability 
to  detect  aircraft  movement  away  from  the  extended  runway  centerline.  However,  for  this 
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simulation,  the  magnification  of  the  controllers’  displays  was  set  at  8  times  for  the  horizontal  axis 
and  2  times  for  the  vertical  axis,  for  a  4: 1  aspect  ratio.  Controllers  were  not  permitted  to  adjust 
the  ratio  during  the  simulation. 

For  each  of  the  runways,  ELS  approach  centerlines  were  displayed  as  dashed  white  lines,  where 
each  dash  and  each  space  between  dashes  represented  1  nm.  Solid  light  blue  lines  were 
displayed  on  each  side  of  the  ILS  centerline  to  delineate  200-ft  deviations  from  the  localizer.  The 
2000-ft  wide  NTZ,  located  equidistant  between  localizer  courses,  was  outlined  in  red. 

A  predictor  line  was  used  in  the  generation  of  the  audio  and  visual  alerts.  The  predictor  line, 
which  was  affixed  to  each  aircraft  target,  indicated  where  the  aircraft  would  have  been  in  10 
seconds  had  it  continued  on  the  same  path.  The  predictor  line  provided  controllers  advance 
notice  of  the  path  of  the  aircraft.  The  predictor  line  can  be  varied  but,  for  this  simulation,  it  was 
set  to  10  seconds. 

Aircraft  targets  and  alphanumeric  data  blocks  were  presented  in  green  as  long  as  aircraft 
maintained,  and  were  predicted  to  maintain,  approaches  within  the  NOZ.  When  a  predictor  line 
indicated  that  an  aircraft  was  within  10  seconds  of  entering  the  NTZ,  the  green  aircraft  target  and 
data  block  changed  to  yellow.  An  auditory  warning  also  sounded  at  that  time  (e.g.,  American 
21 1)  to  notify  controllers  of  the  impending  NTZ  entry.  If  an  aircraft  entered  the  NTZ,  the  yellow 
aircraft  target  and  data  block  immediately  changed  to  red. 

4.2.2  Electronic  Scanning  Radar  Sensor 

The  simulated  Electronic  Scanning  (E-Scan)  sensor  used  a  monopulse  azimuth  measurement 
technique,  which  provided  accuracy  of  better  than  1  milliradian  (0.06  degrees)  root  mean  square 
(rms).  The  range  error  associated  with  the  system  was  ±30  ft  with  an  rms  error  of  25  ft.  The 
specified  system  delay  from  the  antenna  to  the  display  was  up  to  0.5  seconds.  The  alpha-beta 
tracker,  used  to  smooth  aircraft  position  data,  had  gains  of  0.3  (alpha)  and  0.245  (beta)  in  the 
calculation  of  aircraft  positions  and  velocities,  respectively. 

4.2.3  Navigational  Error  Model 

Aircraft  position  with  respect  to  the  final  approach  course,  the  NTZ,  and  other  aircraft,  had  to  be 
realistically  presented  on  the  radar  display  to  accurately  assess  the  controllers'  ability  to  detect 
blunders.  In  developing  the  navigational  error  model  for  TGF  aircraft,  two  criteria  were  used. 
First,  aggregate  errors  had  to  accurately  reflect  the  TNSE  distribution  of  aircraft  as  they  flew  ILS 
approaches.  Second,  displayed  flight  paths  of  aircraft  had  to  look  reasonable  to  the  controllers 
(i.e.,  deviations  from  the  localizer  centerline  had  to  appear  typical  of  aircraft  flying  an  ILS 
approach  during  IMC).  The  navigational  error  model  used  for  this  simulation  was  based  upon 
data  collected  at  Chicago  O'Hare  Airport  (Timoteo  &  Thomas,  1989),  Memphis  Airport  (PRM 
Program  Office,  1991),  and  Los  Angeles  International  Airport  (LAX)  (DiMeo,  Melville, 

Church  well,  &  Hubert,  1993). 
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4.3  Flight  Simulators 


Four  full-motion  air  carrier  simulators  and  one  general  aviation  trainer  (GAT)  were  integrated 
into  the  simulation.  They  assumed  the  configuration  of  aircraft  flying  the  localizer  course  and 
replaced  certain  TGF  aircraft  that  entered  the  simulation.  Table  1  lists  the  participating  simulator 
aircraft. 


Table  1.  Participating  Flight  Simulator  Types,  Sponsoring  Facilities,  and  Locations 


SIMULATOR  TYPE 

SPONSORING  FACILITY  and  LOCATION 

1.  B747-400 

NASA-Ames,  Moffett  Field,  CA 

2.  MD90 

Delta  Airlines  Inc.,  Atlanta,  GA 

3.  B727 

AVIA  Inc.,  Costa  Mesa,  CA 

4.  B727 

Mike  Monroney  Aeronautical  Center;  Oklahoma  City,  OK 

5.  GAT 

William  J.  Hughes  Technical  Center,  Atlantic  City 

International  Airport,  NJ 

Flight  simulators  were  an  integral  part  of  the  real-time  simulation  because  they  provided  a 
representative  sample  of  NAS  users.  The  simulators  also  generated  more  accurate  pilot  and 
aircraft  performance  data  than  the  computer-generated  aircraft. 

The  type  of  approach  flown  by  pilots  of  the  flight  simulators  (i.e.,  coupled  autopilot,  hand-flown 
using  the  flight  director,  raw  data)  was  determined  based  upon  surveys  of  current  airline 
procedures.  For  simulators  that  were  glass  cockpit/FMS-equipped  (NASA  B747-400,  Delta 
MD90),  80  percent  of  the  approaches  were  scripted  to  be  flown  using  a  coupled  autopilot  and  20 
percent  were  to  be  hand-flown  using  the  flight  director.  For  analog/conventional  simulators 
(AVIA  B727,  Oklahoma  City  B727),  50  percent  of  the  approaches  were  scripted  to  be  flown 
using  a  coupled  autopilot  and  50  percent  were  to  be  hand-flown.  The  GAT  was  scripted  such 
that  5  percent  of  its  approaches  were  to  be  flown  using  a  coupled  autopilot,  45  percent  were  to  be 
hand-flown,  and  50  percent  were  to  be  flown  using  raw  data. 

Crosswinds  were  introduced  to  flight  simulator  approaches  to  provide  pilots  with  a  realistic  flight 
environment.  Three  direct  crosswind  conditions  were  assigned:  no  wind,  a  15-kt  wind  from  the 
east,  or  a  15-kt  wind  from  the  west.  All  flight  simulators  were  assigned  the  same  wind  condition 
for  any  given  run. 

5.  Simulation  Instruments 

5.1  Traffic  Samples 

Four  traffic  samples  were  generated  for  the  simulation  based  on  a  survey  of  instrument 
operations  for  several  level  5  (i.e.,  airports  that  have  100  or  more  instrument  operations  per  hour) 
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terminal  radar  approach  control  (TRACON)  facilities.  Traffic  samples  were  lists  of  all  the 
aircraft  arrivals,  which  included  call  signs,  beacon  codes,  aircraft  types,  and  start  times  for 
entering  the  traffic  scenarios.  Approximately  65  aircraft  per  runway  entered  the  simulation 
scenario  per  2-hour  run.  Departing  aircraft  were  also  programmed  into  the  traffic  scenarios  to 
generate  a  more  realistic  ATC  environment.  Approximately  61  aircraft  per  runway  were 
departures  per  2-hour  run. 

A  representative  number  of  air  carriers  (65  percent),  commuters  (30  percent),  and  general 
aviation  aircraft  (5  percent)  targets  constituted  the  traffic  mix.  Air  carriers  (jets)  and  commuters 
(turboprops)  were  assigned  initial  indicated  airspeeds  of  1 80-200  kts.  General  aviation  aircraft 
(props)  were  assigned  initial  indicated  airspeeds  of  130-150  kts.  Speed  overtakes  were  not 
intentionally  scripted  into  the  traffic  samples;  however,  overtakes  did  randomly  occur  throughout 
the  simulation,  and  monitor  controllers  had  to  make  speed  adjustments  when  necessary. 

5.2  Blunder  Scripts 

The  traffic  samples  were  developed  not  only  to  generate  traffic  for  the  simulation  but  also  to 
determine  when  aircraft  would  be  aligned  for  potential  conflicts.  The  traffic  samples  were 
devised  prior  to  the  simulation  and  observed  on  the  radar  scopes  to  determine  the  call  signs  of 
adjacent  aircraft  targets  and  the  times  of  potential  conflicts.  This  information  was  recorded  to 
generate  blunder  scripts.  The  blunder  scripts  noted  aircraft  pairs,  along  with  the  response 
conditions  and  blunder  paths  (see  Section  7.1),  for  the  test  director  to  use  during  simulation  runs. 

5.3  Closest  Point  of  Approach  Prediction  Tool 

The  Closest  Point  of  Approach  (CPA)  prediction  tool  is  a  software  tool  used  by  the  test  director 
during  MPAP  simulations  to  create  potential  at-risk  blunders.  The  CPA  is  defined  as  the 
smallest  slant  range  distance  between  two  aircraft  involved  in  a  conflict.  The  tool  uses  aircraft 
velocities,  headings,  and  degree  of  turn  for  each  aircraft  pair  in  the  real-time  calculation  of  a 
predicted  CPA.  The  tool  also  calculates  the  elapsed  time  until  the  predicted  CPA  is  reached, 
given  an  immediate  execution  of  a  blunder.  All  of  the  CPA  prediction  tool  information  is 
updated  every  second. 

During  this  simulation,  the  CPA  prediction  tool  presented  call  signs  of  predetermined  potential 
blunder  aircraft  pairs  in  a  window  on  the  test  director's  display.  The  window  could  accommodate 
information  on  eight  aircraft  pairs  at  a  time.  The  aircraft  pairs,  which  appeared  in  the  window, 
were  determined  by  the  blunder  scripts;  however,  the  test  director  had  the  capability  to  delete 
scripted  aircraft  pairs  and/or  add  pairs  that  were  not  originally  included  on  the  blunder  scripts. 

6.  Subjects 

6. 1  Final  Monitor  Controllers 

A  total  of  12  air  traffic  controllers  with  experience  in  simultaneous  parallel  approach  operations 
participated  as  test  subjects.  Controllers  were  selected  from  the  following  TRACON  facilities: 
Atlanta,  Pittsburgh,  St.  Louis,  and  Detroit.  All  controllers  were  volunteers  selected  in  agreement 
with  their  NATCA  offices. 
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Controllers  were  scheduled  to  participate  in  two  groups,  one  group  of  six  participating  each 
week.  The  first  group  participated  October  16-20, 1995,  and  the  second  group  participated 
October  23-27,  1995.  Individual  controllers  were  scheduled  to  work  as  monitor  controllers  for 
half  of  each  2-hour  run.  A  controller  rotation  period  occurred  at  the  midpoint  of  each  2-hour  run 
to  simulate  actual  work  rotations  and  to  give  monitor  controllers  a  rest.  Each  controller 
participated  in  only  1  hour  of  each  2-hour  run.  Furthermore,  controllers  were  not  scheduled  to 
participate  in  more  than  three  runs  on  any  day  of  the  simulation. 

6.1.1  Controller  Briefing 

Controllers  were  given  a  briefing  prior  to  participating  in  the  simulation.  The  briefing  included  a 
description  of  the  MPAP  TWG’s  composition  and  program  goals.  The  simulation  purpose  was 
also  explained,  followed  by  a  presentation  of  a  PRM  video  shown  to  familiarize  controllers  with 
the  components  of  the  PRM  system.  The  briefing  package  included  diagrams  of  the  simulated 
airport  approach  area  configuration  and  the  approach  plates  that  were  contained  in  the  pilot 
briefings.  Controller  schedules  of  participation  were  also  included.  Appendix  D  contains  the 
complete  briefing  as  distributed  to  the  test  controllers. 

The  primary  focus  of  the  briefing  package  was  on  controller  responsibility.  Controllers  were 
given  an  overview  of  the  responsibilities  of  the  final  monitor  controller,  which  included 
maintaining  longitudinal  separation  and  lateral  separation  between  aircraft  on  adjacent  localizers 
and  ensuring  each  aircraft  established  contact  with  the  tower  prior  to  the  first  simultaneous 
approach  fix.  Controllers  were  told  not  to  make  speed  adjustments  to  aircraft  inside  the  final 
approach  fixes  and  were  reminded  of  the  monitor  controller’s  override  capabilities  on  the  local 
control  frequencies. 

The  briefer  read  a  paragraph  from  the  Airman’s  Information  Manual  (FAR,  1995)  to  the 
controllers  that  stated  that  the  primary  navigation  responsibility  was  to  rest  with  the  pilot,  and 
controller-issued  instmctions  were  therefore  limited  to  those  necessary  to  ensure  separation 
between  aircraft  and  to  contain  the  aircraft  flight  paths  within  the  NOZ.  Aircraft  that  were 
observed  to  enter  the  NTZ  were  to  be  instructed  to  alter  course  left  or  right,  as  appropriate,  to 
return  to  the  desired  courses.  Unless  altitude  separation  was  assured,  controllers  monitoring  the 
adjacent  localizer  courses  were  to  take  immediate  action  to  require  aircraft  in  potential  conflict  to 
alter  flight  paths  to  avoid  deviating  aircraft. 

Controllers  were  instructed  to  use  the  following  phraseology  in  the  event  that  an  aircraft  overshot 
the  turn-on  or  continued  on  a  flight  path  that  would  penetrate  the  NTZ: 

•  "You  have  crossed  the  final  approach  course.  Turn  (left/right)  immediately  and  return  to 
the  localizer/azimuth  course,"  or 

•  "Turn  (left/right)  and  return  to  the  localizer/azimuth  course." 
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Controllers  were  instructed  to  use  the  following  phraseology  if  an  aircraft  on  an  adjacent 
approach  was  in  potential  conflict  with  a  deviating  aircraft:  TRAFFIC  ALERT,  aircraft  call  sign, 
turn  (left/right)  immediately  heading  (degrees),  climb  and  maintain  (altitude).  (Proposed  change 
to  FAA  Order  71 10.65) 

In  addition,  controllers  were  instructed  to  use  the  following  standard  breakout  headings  and 
altitudes,  whenever  feasible,  for  aircraft  on  adjacent  courses  to  deviating  aircraft: 

•  Runway  18R:  Turn  right  immediately  heading  two  seven  zero,  climb  and  maintain  five 
thousand. 

•  Runway  18L:  Turn  left  immediately  heading  zero  niner  zero,  climb  and  maintain  four 
thousand. 

6. 1 .2  Controller  Training 

Controllers  were  given  hands-on  training  with  the  PRM  equipment  following  the  briefing  and 
prior  to  actual  test  runs.  Each  group  of  controllers  rotated  through  monitor  positions  over  the 
course  of  five  2-hour  practice  runs.  The  purpose  of  the  training  was  to  familiarize  the  controllers 
with  the  FMA  displays  and  to  expose  them  to  blunder  situations.  Controllers  were  encouraged  to 
practice  the  standard  phraseology  and  to  coordinate  actions  with  other  monitor  controllers  during 
blunders. 

6.2  Pilots 

A  total  of  51  pilots  participated  in  flight  simulators  during  the  simulation.  Of  these,  41  were  air 
carrier  pilots  with  an  average  of  1 1,658  total  flight  hours,  and  10  were  general  aviation,  military, 
or  commuter  pilots  with  an  average  of  2,850  total  flight  hours.  Air  carrier  pilots  recmited  to 
participate  in  the  simulation  were  required  to  be  qualified  in  type  and  current  on  the  aircraft 
represented  by  the  simulator  to  which  they  were  assigned.  Pilots  recmited  to  fly  the  GAT  were 
required  to  hold  commercial  flight  certificates  with  multi-engine  and  instmment  ratings. 

Three  pilots  were  assigned  to  each  air  carrier  flight  simulator  site  each  day,  except  at  the  NASA 
facility  where  two  pilots  flew  together  the  entire  day.  Single  pilot  IFR  operations  were  conducted 
at  the  GAT,  therefore,  two  pilots  divided  the  flying  time  each  day.  Each  pilot  flew  approximately 
8  approaches  in  the  air  carrier  simulators  and  approximately  10  approaches  in  the  GAT  each  day. 

6.2. 1  Pilot  Briefing  and  Training 

Pilots  reported  1  hour  before  the  start  of  the  simulation  for  training.  After  reviewing  a  Pilot 
Briefing  Handout  (Appendix  E),  pilots  were  shown  a  12-minute  video  describing  the  PRM 
system  (FAA,  1995c).  Following  the  video,  pilots  were  given  approach  plates  and  an  airport 
information  page.  They  were  required  to  read  a  Pilot  Awareness  Training  Bulletin  (Appendix  E) 
and  take  a  self-administered  test  to  reinforce  what  they  had  read.  Pilots  assigned  to  the  glass 
cockpit  simulators  (i.e.,  B747-400  and  MD90)  were  required  to  read  a  Breakout  Procedure 
Bulletin  and  take  a  self-administered  test  on  the  material.  This  bulletin  presented  a  hand-flown 
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breakout  procedure  that  emphasized  turning  off  the  flight  director  of  the  pilot  flying  until  the 
pilot  not  flying  changed  the  flight  director  inputs  to  conform  with  the  breakout  instructions. 

This  training  session  was  designed  to  be  similar  to  what  a  pilot  could  encounter  at  an  airline. 
After  reporting  for  work,  the  pilot  would  find  the  training  bulletins  in  his  or  her  mailbox  along 
with  self-administered  tests.  The  pilot  would  read  the  bulletins,  complete  the  tests,  and  hand 
them  in  to  the  chief  pilot.  The  pilot's  training  records  would  be  updated  accordingly.  The  Pilot 
Briefing  Handout,  the  Pilot  Awareness  Training  Bulletin,  the  Breakout  Procedure  Bulletins,  and 
the  corresponding  self-administered  tests  are  located  in  Appendix  E. 

6.2.2  Pilot  Briefing  Materials 

6.2.2. 1  Approach  Information  Index  Cards 

Prior  to  each  approach,  the  pilots  were  handed  an  Approach  Information  Index  Card  to  place  on 
the  cockpit  console.  These  cards  provided  simulator  pilots,  site  coordinators  (see  Section  8.6), 
and  technicians  with  the  necessary  details  to  insure  that  the  simulators  were  in  the  proper 
configuration  at  the  initial  point  before  entering  the  simulation.  The  following  information  was 
included  on  the  index  card: 

a.  runway, 

b.  type  aircraft  and  equipment  displayed  on  FMA  display, 

c.  initial  point  time  at  which  the  aircraft  was  scheduled  to  enter  the  simulation, 

d.  aircraft  call  sign  (located  in  center  of  card  in  large  font  for  ease  of  recognition), 

e.  initial  heading  to  fly, 

f.  initial  altitude, 

g.  initial  indicated  airspeed, 

h.  localizer  frequency, 

i.  tower  frequency, 

j.  transponder  code, 

k.  type  of  approach  in  accordance  with  percentage  based  on  survey  of  airlines, 

l.  traffic  sample  number, 

m.  site  corresponding  to  each  set  of  index  cards  and  traffic  sample  number,  and 

n.  index  number. 

6.2.2.2  Approach  Plates  and  Airport  Information  Page 

Pilots  were  supplied  with  approach  plates  designed  specifically  for  simultaneous  close  parallel 
approaches,  along  with  an  airport  information  page.  The  approach  plates  differed  from  those 
used  for  normal  ILS  approaches  in  that  they  included  information  designed  to  heighten  pilot 
awareness  of  close  parallel  ILS  operations.  Three  notes  were  placed  in  the  plan  view  section  of 
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the  approach  plates.  The  first  note  identified  the  altitude  below  which  no  descending  breakouts 
would  be  issued.  The  second  note  authorized  simultaneous  close  parallel  approaches  with  the 
adjacent  runway.  The  third  note  required  the  pilot  to  read  the  airport  information  page  before 
flying  the  close  parallel  approach.  In  the  heading  section  of  the  approach  plates,  under  the 
procedure  identification,  the  words  Close  Parallel  were  added. 

The  airport  information  page  contained  an  illustration  of  the  centerline  spacing  between  runways, 
the  pilot  requirements  for  flying  the  simultaneous  close  parallel  approach,  a  paragraph  on 
breakout  descents,  a  section  on  controller  phraseology,  and  emphasis  on  the  importance  of  an 
immediate  pilot  response  to  a  controller's  breakout  instructions.  The  approach  plates  and  airport 
information  page  are  located  in  Appendix  F. 

6.2.2.3  Automatic  Terminal  Information  System  Cards 

In  order  to  maintain  a  realistic  environment  during  the  simulation,  pilots  were  provided  with 
written  automatic  terminal  information  system  (ATIS)  cards.  These  cards  acted  as  the  ATIS 
broadcast  the  pilots  would  listen  to  before  entering  the  airport  environment.  The  ceiling, 
visibility,  restrictions  to  visibility,  temperature,  dewpoint,  wind,  altimeter  setting,  approaches  in 
use,  and  letter  identifier  were  written  on  the  cards.  Three  separate  cards  were  used  to  reflect  the 
changing  wind  conditions  used  in  the  simulation.  These  cards  were  identified  as  information 
alpha,  information  bravo,  and  information  charlie,  respectively.  An  example  of  the  ATIS  cards  is 
contained  in  Appendix  E. 

7.  Experimental  Design 

7. 1  Experimental  Factors  Description 

All  blundering  aircraft  in  the  simulation  were  scripted  to  have  certain  response  conditions 
(responding  or  nonresponding)  and  blunder  paths  (maintain  altitude  or  descend).  In  addition, 
blunders  were  distributed  along  the  localizer  courses  and  initiated  towards  certain  types  of 
aircraft  according  to  predetermined  percentages. 

7.1.1  Response  Condition 

To  simulate  worst-case  situations,  where  blundering  aircraft  were  unable  to  correct  their 
deviations,  pilots  of  blundering  aircraft  were  often  instructed  to  disregard  controller 
communications,  thereby  not  correcting  the  blunder.  A  blunder  with  this  condition  was  referred 
to  as  a  nonresponding  blunder.  Throughout  the  simulation,  80  percent  of  the  blundering  aircraft 
were  scripted  to  be  nonresponding. 

7.1.2  Blunder  Path 

For  additional  realism,  the  paths  of  blundering  aircraft  were  scripted  to  either  maintain  altitude  or 
descend  on  a  3-degree  glidepath.  Fifty  percent  were  scripted  for  each  condition. 
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7. 1 .3  Blunder  Distribution  Along  Localizer  Course 


For  purposes  of  tracking  where  blunders  occurred  along  the  localizer  courses,  distances  from  the 
runway  thresholds  were  binned  into  the  following  groups:  1-3  nm,  3-5  nm,  5-7  nm,  7-9  nm,  and 
9-12  nm.  The  goal  was  to  have  18  percent  of  the  blunders  occur  in  each  bin,  except  for  the  9-12 
nm  bin,  which  had  a  blunder  initiation  goal  of  28  percent. 

7. 1 .4  Aircraft  Types  and  Flight  Systems 

Blunders  were  also  initiated  according  to  predetermined  traffic  mix  percentages  based  upon 
aircraft  types  and  flight  systems.  Table  2  summarizes  the  blunder  initiation  traffic  mix 
percentage  goals.  See  Appendix  G  for  a  summary  of  all  of  the  simulation  distribution  goals  and 
actual  results. 


Table  2.  Blunder  Initiation  Goals  for  Aircraft  Types  and  Flight  Systems 


Flight  Simulator 

Goal 

(percent) 

Evading  Aircraft  Type 

Heavy  Jet 

B747-400 

30 

Jet 

B727s,  MD90 

65 

General  Aviation 

GAT 

5 

Evading  Aircraft  Flight  System 

Glass 

B747-400,  MD90 

60 

Analog/Conventional 

B727s 

35 

General  Aviation 

GAT 

5 

7.2  Performance  Measures 

Dependent  variables  in  the  simulation  included  CPA,  frequency  of  NTZ  entries,  and  frequency  of 
NBOs. 

7.2. 1  Closest  Point  of  Approach 

Blunder  resolution  performance  was  measured  by  determining  the  proportion  of  successfully 
resolved  conflicts  relative  to  the  total  number  of  blunders  that  would  have  resulted  in  TCVs  had 
there  been  no  controller  intervention.  The  resolution  of  conflicts  was  examined  by  calculating 
CPAs.  The  CPA  was  the  smallest  slant  range  distance  between  two  aircraft  involved  in  a  conflict 
(measured  in  ft).  The  distance  was  measured  from  the  center  of  each  aircraft  involved  in  the 
conflict  and  was  updated  every  second. 

7.2.2  Frequency  of  No  Transgression  Zone  Entries  and  Nuisance  Breakouts 

The  number  of  NTZ  entries  and  NBOs  was  determined  and  used  as  a  measure  of  system  capacity. 
Frequencies  of  NTZ  entries  and  NBOs  were  computed  through  review  of  PRM  video  and  audio 
recordings  of  each  run  of  the  simulation. 
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7.3  Procedure 


Controllers  staffed  two  final  approach  monitor  positions.  They  monitored  the  flight  paths  of  the 
aircraft  on  their  assigned  runways  and  their  task  was  to  ensure  that  aircraft  maintained  the 
required  separation.  Aircraft  blunders  were  initiated  to  test  the  ability  of  the  ATC  system  to 
maintain  adequate  distances  between  aircraft  during  critical  situations.  During  each  run  of  the 
simulation,  blunders  occurred  without  warning  to  the  controllers.  During  blunder  events, 
controllers  issued  control  instructions  to  attempt  to  resolve  the  situation. 

Blunder  scripts,  traffic  samples,  and  the  CPA  prediction  tool  guided  the  initiation  of  the  blunders. 
Approximately  16  blunders  were  scripted  per  2-hour  run.  Blunders  did  not  occur  within  less  than 
3  minutes  of  each  other  or  within  1  nm  of  the  runway  thresholds.  All  blundering  aircraft  were 
TGF  aircraft  and  most  evading  aircraft  were  flight  simulators. 

8.  Support  Personnel 

8.1  Test  Director 


Simulation  runs  and  aircraft  blunders  were  initiated  at  the  instruction  of  the  simulation  test 
director.  Individuals  who  assumed  the  role  of  test  director  had  extensive  ATC  experience  and 
were  trained  to  work  with  the  CPA  prediction  tool.  The  test  director  was  responsible  for 
initiating  blunders  based  upon  the  information  provided  by  the  blunder  scripts,  the  CPA 
prediction  tool,  and  his  expert  judgment. 

8.2  Controller  Technical  Observers 

Four  controller  technical  observers  participated  in  the  simulation,  all  of  whom  had  ATC 
experience  and  were  familiar  with  the  MPAP  project.  Controller  technical  observers  monitored 
controller  actions  during  each  simulation  run.  Their  duties  included  documenting  discrepancies 
between  issued  control  instructions  and  actual  aircraft  responses,  alerting  responsible  parties  to 
any  problems  that  may  have  occurred  during  the  test  (e.g.,  computer  failure,  stuck  microphone), 
assisting  controllers  with  the  preparation  of  blunder  statements,  and  preparing  a  controller 
technical  observer  assessment  at  the  end  of  the  simulation  (see  Section  1 1.6.1).  The  assessment 
included  their  opinions  and  conclusions  concerning  the  conduct  of  the  simulation  as  well  as  any 
recommendations  to  the  MPAP  TWG. 

8.3  Pilot  Technical  Observers 

A  group  of  former  air  carrier  pilots  with  experience  in  numerous  air  carrier  aircraft,  including 
several  thousand  hours  in  Boeing  glass  cockpits,  participated  as  pilot  technical  observers  in  the 
simulation.  They  had  acted  as  site  coordinators  in  previous  simulations  and  were  involved  in  the 
development  of  the  pilot  training  used  in  this  simulation.  They  observed  the  3000-ft  simulation 
from  several  viewpoints,  including  the  simulator  cockpits,  the  test  director's  station,  and  the 
visitor's  observation  room.  In  addition,  they  reviewed  videotapes  of  breakouts  from  all  of  the 
simulator  sites. 
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8.4  Simulation  Pilot  Operators 


SPOs  operated  the  TGF  aircraft  during  simulation  runs.  SPOs  controlled  blundering  aircraft  at 
the  instruction  of  the  test  director  and  responded  to  controller  instructions  (except  during 
nonresponding  blunders)  by  entering  aircraft  heading  and/or  altitude  changes  using  their 
specialized  computer  keyboards  and  displays. 

8.5  Tower  Controllers 

To  add  realism  to  the  communications  on  the  final  monitor  frequencies,  three  non-subject  tower 
controllers  rotated  through  performing  local  tower  control  functions.  They  cleared  aircraft  for 
departure  and  landing  and  advised  frequency  changes. 

8.6  Site  Coordinators 

Site  coordinators  were  assigned  to  each  flight  simulator  location  to  coordinate  efforts  with  the 
test  director  at  the  William  J.  Hughes  Technical  Center  and  to  support  pilots  during  their 
participation  in  the  simulation.  They  were  provided  with  a  Site  Coordinator  Briefing  Materials 
package  that  detailed  their  duties  and  responsibilities.  Site  coordinators  were  current  or  retired 
airline  pilots  who  had  experience  with  MPAP  real-time  simulations  and  with  the  type  of  aircraft 
represented  by  the  flight  simulator  to  which  they  were  assigned. 

Site  coordinators  acted  as  observers  and  were  trained  to  refrain  from  providing  any  help  to  the 
aircrews  that  would  invalidate  the  simulation  data.  Their  responsibilities  included  briefing 
aircrews,  providing  pilots  with  flight  information  prior  to  each  approach,  documenting  approach 
information,  and  administering  questionnaires  to  the  pilots.  The  Site  Coordinator  Briefing 
Materials  are  located  in  Appendix  H. 

8.7  Simulation  Observer 

A  simulation  observer  manually  documented  information  from  the  test  director's  station, 
including  blunder  occurrences,  NBOs,  NTZ  entries,  potential  TCVs,  lost  beacon  signals  (i.e., 
aircraft  that  went  into  coast  (CST),  indicating  a  loss  of  radar  tracking),  and  system  problems 
(e.g.,  communications  failure,  hardware/software  failure). 

9.  Data  Collection 

9. 1  Computer-Generated  Data  Files 

The  generation  of  data  files  by  the  TGF  allowed  a  detailed  examination  of  the  performance  of  the 
ATC  system  in  resolving  blunders.  Data  files  included  information  on  parallel  conflict 
frequencies,  parallel  conflict  slant  range  miss  distances,  and  aircraft  position/track  data. 

Data  files  were  also  generated  at  the  flight  simulator  sites.  These  files  contained  detailed 
information  about  aircraft  performance  (e.g.,  angle  of  bank,  rate  of  climb,  pitch  angle),  allowing 
detailed  analysis  of  pilot/aircraft  response. 
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9.2  Audio  and  Video  Recordings 


The  communication  frequencies  of  both  blundering  and  evading  aircraft  and  visual  components 
of  the  PRM  display  throughout  each  run  were  recorded  on  a  Super- VHS  video  cassette  recorder. 
Backup  audio  recordings  were  generated  using  a  20-channel  DICTAPHONE  audio  recorder  and 
a  9-channel  IONICA  audio  recorder.  Both  the  DICTAPHONE  and  the  IONICA  systems  were 
recorded  from  the  AMECOM  system  and  operated  independently  of  one  another  and  of  the  TGF 
operating  system. 

Video  cameras  were  set  up  in  all  flight  simulators  to  capture  the  interactions  between  the  pilots  in 
the  cockpit  and  between  pilots  and  controllers.  In  addition,  a  video  camera  was  mounted  behind 
the  controllers  in  the  monitor  room  to  capture  all  interactions  and  coordinating  efforts  between 
controllers  during  blunders  and  other  events  in  the  simulation. 

The  videotapes  were  used  in  the  examination  of  TCVs,  the  evaluation  of  controller  phraseology 
and  other  message  characteristics,  the  extraction  of  controller  and  pilot  response  times,  the 
identification  of  NTZ  entries  and  NBOs,  and  the  verification  of  computer-generated  data  file 
information. 

9.3  Questionnaires 

9.3.1  Controller  Questionnaires 

At  the  end  of  each  day,  controllers  completed  End-of-Day  Controller  Questionnaires  that 
addressed  the  level  of  activity,  stress,  and  mental  effort  they  experienced  throughout  the  day.  In 
addition,  when  the  controller  technical  observers  believed  a  TCV  had  occurred,  they  instructed 
controllers  to  describe  the  conflict  in  detail  on  a  Blunder  Statement.  Controllers  also  completed  a 
Post-Simulation  Questionnaire  at  the  conclusion  of  their  participation  in  the  simulation.  The 
Post-Simulation  Questionnaire  addressed  issues  such  as  the  operational  viability  of  the  runway 
configuration,  the  degree  of  communications  workload,  and  simulation  realism.  The  controller 
questionnaires  are  found  in  Appendix  I. 

9.3.2  Pilot  Questionnaires 

Two  pilot  questionnaires  were  employed  in  this  simulation.  After  every  breakout,  site 
coordinators  used  a  Pilot  Breakout  Questionnaire  to  evaluate  the  pilot's  performance  from  initial 
controller  transmission  to  the  resultant  breakout  maneuver.  Second,  flight  simulator  pilots 
completed  a  Flight  Crew  Opinion  Survey  at  the  conclusion  of  their  participation  in  the 
simulation.  The  Flight  Crew  Opinion  Survey  collected  subjective  data  on  such  issues  as  the 
adequacy  of  the  training  materials,  approach  plates,  information  page,  and  breakout  instructions. 
Both  flight  simulator  pilot  questionnaires  can  be  found  in  Appendix  J. 

9.4  Observer  Logs 

Controller  technical  observers  and  site  coordinators  each  recorded  information  on  logs  designed 
specifically  for  their  tasks.  In  general,  the  controller  technical  observer  logs  were  designed  to 
capture  information  pertaining  to  blunders,  potential  TCVs,  NBOs,  NTZ  entries,  and  simulation 
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problems  to  be  used  in  conjunction  with  the  computer-generated  data.  Site  coordinator  logs  were 
developed  to  record  approach  and  breakout  information,  such  as  approach  identification, 
simulator  problems,  approach  abnormalities,  answers  to  the  Pilot  Breakout  Questionnaires,  and 
comments  from  the  observer  or  the  pilot  concerning  the  approach.  Examples  of  the  observer  logs 
are  located  in  Appendix  K. 

10.  Assessment  Methodology 

All  data  collection  sources,  including  computer-generated  data,  video  and  audio  data,  pilot  and 
controller  questionnaires,  and  observer  logs  were  used  to  evaluate  the  proposed  operation 
performance  in  meeting  the  established  test  criteria. 

Only  data  from  blunders  involving  flight  simulators  as  evading  aircraft  were  used  in  the  blunder 
resolution  performance  and  NTZ  entry  analyses  due  to  aerodynamic  performance  differences  that 
have  been  identified  between  TGF  aircraft  and  flight  simulators.  The  TGF  interface  does  not 
enable  SPOs  to  respond  in  a  manner  that  is  representative  of  operational  aircraft.  In  addition, 
SPOs  are  not  actual  line  pilots.  Both  flight  simulator  and  TGF  aircraft  were  included  in  the  NBO 
analysis,  since  NBOs  were  usually  caused  by  TNSE,  and  the  fidelity  of  the  pilot/aircraft 
performance  was  not  as  critical  as  in  the  blunder  resolution  performance. 

Two  Monte  Carlo  computer  simulations  of  parallel  approach  blunders  were  used  after  the  real¬ 
time  simulation  to  enhance  the  risk  assessment  part  of  the  analysis.  The  test  condition 
parameters  were  the  same  as  for  the  real-time  simulation.  The  Monte  Carlo  analyses  used  the 
recorded  controller  and  aircraft  response  data  as  inputs  to  the  models.  The  Monte  Carlo  TCV 
rate  results  were  compared  to  the  real-time  TCV  rate  results  to  ensure  they  were  compatible.  The 
similarities  and  differences  between  the  Monte  Carlo  simulations  are  detailed  in  Appendix  B,  and 
a  description  of  the  risk  assessment  methodology  can  be  found  in  Appendix  C. 

In  addition  to  the  data  available  from  the  simulation,  the  MPAP  TWG  drew  upon  their 
understanding  of  the  nature  of  daily  operations,  the  knowledge  and  skills  of  controllers  and 
pilots,  and  the  full  range  of  traffic  contingencies  to  evaluate  the  pilot  and  controller 
communications  workload  and  to  develop  their  operational  assessment  of  the  proposed  operation. 

11.  Simulation  Results 

It  should  be  kept  in  mind  that,  like  the  output  of  all  experimental  evaluations,  the  results  of  this 
study  should  not  be  extrapolated  to  situations  that  contain  variables  other  than  those  tested  in  this 
study. 

11.1  Test  Criterion  Violation  Review 

No  TCVs  occurred  in  the  real-time  simulation.  One  blunder  resulted  in  an  aircraft  miss  distance 
of  less  than  500  ft,  but  was  deemed  invalid  due  to  faulty  equipment.  The  controller  issued 
breakout  instructions  several  times,  but  nothing  was  transmitted  on  the  frequency.  The 
equipment  problem  was  verified  through  video  and  audio  review  of  the  controller  monitor  room, 
the  PRM  display,  and  the  flight  simulator  site  involved  in  the  blunder. 
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1 1 .2  Test  Criterion  Violation  Rate  and  Risk  Analyses 


Three  analyses  were  conducted  to  estimate  the  TCV  rate.  First,  the  real-time  simulation  data 
were  used  in  the  calculation  of  an  at-risk  TCV  rate.  Then,  two  fast-time  Monte  Carlo 
simulations,  the  BRM  and  the  AS  AT,  were  conducted  to  increase  the  sample  size  using  the  real¬ 
time  data,  thus  providing  more  accurate  estimates  of  the  TCV  rate.  The  confidence  intervals 
were  determined  for  each  of  the  TCV  rates  and  were  compared  to  the  test  criterion  rate  of  6.8 
percent.  These  analyses  are  discussed  in  the  following  sections. 

11.2.1  Real-Time  Simulation 

Out  of  a  total  of  304  blunders  initiated  over  the  course  of  the  real-time  simulation,  284  were 
considered  at-risk:  253  nonresponding  and  31  responding.  The  observed  TCV  rate  from  the  real¬ 
time  simulation  was  0.0  percent  (0  TCVs/253  at-risk,  nonresponding  blunders).  The  99-percent 
confidence  interval  was  0.0  to  2.1  percent.  Thus,  the  population  TCV  rate  would  very  likely  be 
below  the  test  criterion  of  6.8  percent. 

1 1.2.2  Monte  Carlo  Simulations 

The  following  sections  report  the  results  of  the  BRM  and  the  ASAT  Monte  Carlo  simulations 
and  how  they  compared  to  the  maximum  acceptable  TCV  rate.  Although  the  Monte  Carlo 
models  used  similar  methodologies,  each  model  had  its  own  unique  inputs  and  modeling 
techniques,  which  may  have  accounted  for  differences  in  the  results.  For  details  on  the  BRM  and 
ASAT  model  configurations,  see  Appendix  B. 

1 1 .2.2. 1  Blunder  Risk  Model  Results 

The  BRM  Monte  Carlo  simulation  executed  125,490  at-risk,  nonresponding  blunders  that 
resulted  in  a  TCV  rate  of  1.9  percent.  The  99-percent  confidence  interval  was  1.8  to  2.0  percent. 
The  BRM  Monte  Carlo  TCV  rate  was  also  below  the  test  criterion  maximum  TCV  rate  of  6.8 
percent. 

The  BRM  TCV  rate  was  calculated  as  a  function  of  distance  from  the  threshold  for  additional 
analysis  purposes.  Since  the  approach  courses  were  offset  radially,  it  was  expected  that  the  TCV 
rate  would  be  greater  closer  to  the  runway  threshold.  The  results  are  shown  in  Table  3.  All 
values  were  still  well  below  the  maximum  acceptable  TCV  rate  of  6.8  percent. 

Table  3.  Blunder  Risk  Model  Test  Criterion  Violation  Rates  as  a  Function  of  Distance  from 

Runway  Threshold 


Range  (nm) 

9.5, 10.5, 11.5 

7.5,  8.5 

5.5,  6.5 

3.5, 4.5 

1.5,  2.5 

Number  of  At-Risk 
Blunders 

35,294 

23,529 

23,529 

23,529 

23,529 

TCV  Rate  (percent) 

0.9 

0.9 

3.1 

2.8 

2.2 
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Analysis  of  the  BRM  trials  that  resulted  in  TCVs  showed  that  1 1  evader  tracks  were  responsible 
for  70  percent  of  the  TCVs:  1  each  in  the  range  bins  with  0.9  percent  TCV  rates  and  3  each  in 
the  other  range  bins.  These  tracks  were  analyzed  to  determine  the  contributing  factors  that 
caused  these  pilot/aircraft  evader  responses  to  be  less  effective.  These  tracks  shared  a 
combination  of  the  following  traits:  late  and/or  shallow  response,  aircraft  motion  towards  the 
NTZ,  and  first  transmission  not  received,  thereby  delaying  the  evader  response. 

11.2.2.2  AS  AT  Results 

The  ASAT  Monte  Carlo  simulation  executed  427,978  at-risk,  nonresponding  blunders  that 
resulted  in  a  TCV  rate  of  0.1 1  percent.  The  99-percent  confidence  interval  was  0.10  to  0.12 
percent.  The  ASAT  Monte  Carlo  TCV  rate  was  also  below  the  test  criterion  maximum  TCV  rate 
of  6.8  percent. 

The  ASAT  TCV  rate  was  also  calculated  as  a  function  of  distance  from  the  runway  threshold. 
Table  4  shows  the  results  of  this  additional  analysis.  The  ASAT  TCV  rates  remained  well  below 
the  maximum  acceptable  TCV  rate  of  6.8  percent  at  all  points  on  the  approach. 

Table  4.  ASAT  Test  Criterion  Violation  Rates  as  a  Function  of  Distance  from  Runway  Threshold 


10 

8 

6 

5 

4 

3 

2 

Number  of  At- 
Risk  Blunders 

79,433 

53,830 

51,735 

59,300 

61,200 

56,900 

65,580 

TCV  Rate 
(percent) 

0.014 

0.050 

0.070 

0.080 

0.120 

0.190 

0.247 

The  TCV  rate  estimates  obtained  from  the  Monte  Carlo  simulations  corresponded  with  and  thus 
supported  the  results  obtained  from  the  real-time  simulation.  Based  upon  these  findings,  the 
conduct  of  the  tested  procedure  should  not  increase  the  risk  of  an  accident  above  the  target  level 
of  safety. 

11.3  No  Transgression  Zone  Entry  and  Nuisance  Breakout  Analyses 

No  NTZ  entries  were  made  by  flight  simulators  that  were  not  the  result  of  a  blunder  or  breakout. 
Therefore,  the  NTZ  entry  rate  was  acceptable. 

A  total  of  22  NBOs  occurred  in  the  real-time  simulation  out  of  the  1,724  nonblunder-related 
approaches  (1.3  percent).  The  TWG  and  technical  observers  agreed  that  the  NBO  rate  was  at  an 
acceptable  level. 

1 1 .4  Controller  Communications  Workload 

The  controller  communications  workload  associated  with  TNSE-related  events  for  the  proposed 
operation  was  deemed  satisfactory  by  participating  controllers,  controller  technical  observers, 
and  the  TWG.  Since  the  number  of  NTZ  entries  and  NBOs  was  not  excessive,  controllers  were 
not  overburdened  with  communications  to  aircraft  flying  the  approaches. 
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1 1 .5  Technical  Work  Group  Operational  Assessment 


Based  upon  the  test  results,  the  TWG  expertise  and  judgment,  and  evaluations  from  the  controller 
and  pilot  technical  observers,  the  TWG  unanimously  agreed  that  the  procedure  met  all  of  the 
established  criteria.  The  enhanced  pilot  and  controller  training  procedures  and  the  modified 
controller  breakout  phraseology  clearly  contributed  to  the  success  of  the  proposed  operation. 

Pilots  were  better  trained  on  situational  awareness  and  flight  procedures  than  in  the  previous 
3000-ft  simulation.  In  addition,  controllers  were  better  trained  on  the  use  of  the  breakout 
phraseology  and  in  the  use  of  the  FMA  than  in  the  previous  simulation.  The  modified 
phraseology  itself  helped  to  reduce  the  number  of  blocked  and  clipped  communications  with  the 
addition  of  the  words  "Traffic  Alert"  prior  to  the  call  sign  announcement.  The  enhanced  training 
procedures  and  phraseology  affected  overall  system  performance  so  that  responsibilities  were 
clearly  understood,  response  times  were  improved,  and  adequate  aircraft  separation  was 
maintained.  Although  the  ABDs  were  not  used  in  this  simulation,  they  will  be  required  in 
ground  radios  before  implementation  of  this  close  parallel  configuration.  In  the  event  that  ABDs 
are  not  available,  the  air  traffic  facility  will  have  two  communication  frequencies  available  for 
each  close  parallel  runway,  allowing  the  tower  and  monitor  controllers  to  simultaneously 
broadcast  on  both  frequencies  to  reduce  the  effect  of  blocked  communications. 

Although  not  tested  in  this  simulation,  the  TWG  recognized  the  need  to  address  the  missed 
approach  procedure  for  the  closely  spaced  operation.  A  separate  study  was  done  to  evaluate  the 
missed  approach,  which  is  described  in  Appendix  M.  As  a  result  of  the  study,  the  TWG 
recommended  that  controller  monitoring  of  the  missed  approach  be  required  to  ensure  adequate 
separation  of  aircraft  in  the  event  of  a  simultaneous  missed  approach  on  the  3000  ft,  2.5-degree 
offset  configuration. 

The  TWG  unanimously  agreed  to  recommend  the  tested  procedure  on  simultaneous  approaches 
to  dual  runways  spaced  3000  ft  apart  using  the  PRM  system  with  a  1.0  second  update  rate  and  a 
2.5-degree  localizer  offset  for  approval  in  the  operational  environment. 

11.6  Additional  Analyses 

The  TWG  had  additional  data  available  to  them  to  refer  to  as  necessary  when  evaluating  the  four 
test  criteria.  The  following  sections  describe  in  detail  the  supplemental  assessments  and  analyses 
that  were  performed. 

11.6.1  Controller  Technical  Observer  Assessment 

During  the  simulation,  the  controller  technical  observers  witnessed  improvements  in  the  problem 
areas  identified  during  the  previous  3000-ft  simulation.  Those  problem  areas  were  aircraft 
response  times,  communications,  and  phraseology. 

Aircraft  response  times  appeared  to  be  faster.  This  may  have  been  due  to  a  combination  of 
heightened  awareness  training  and/or  hand-flown  breakout  maneuvers.  Regardless,  the  response 
times  appeared  to  decrease  significantly.  Improved  phraseology  benefited  the  simulation  in  the 
area  of  communications  and  phraseology.  The  phrase  "Traffic  Alert"  at  the  beginning  of  the 


26 


transmission  not  only  alerted  all  pilots  on  the  frequency  to  listen  for  an  urgent  message,  but  also 
ensured  that  if  the  beginning  of  the  transmission  was  blocked,  the  aircraft  call  sign  could  still  be 
heard. 

The  controller  technical  observers  believed  that  improved  controller  training,  as  well  as  improved 
phraseology,  played  a  significant  role  in  the  success  of  the  simulation.  Prior  to  participation  in 
actual  test  runs,  controllers  attended  a  briefing  consisting  of  information  on  the  PRM  system,  the 
simulated  airport  environment  (including  airport  layout,  approaches,  and  minimum  vectoring 
altitudes),  and  controller  responsibilities.  Also  included  in  the  briefing  was  an  aircraft  simulator 
cockpit  video,  which  was  shown  to  educate  the  controllers  on  what  occurs  after  breakout 
maneuvers  are  issued.  The  briefing  was  instrumental  in  helping  the  controller  make  the  most 
effective  decision  when  issuing  the  breakout  instruction,  especially  when  a  descending  breakout 
instruction  was  involved.  In  addition  to  knowledge  gained  from  the  briefing,  8  hours  on  position 
and  resolving  blunders  were  required  for  each  controller  prior  to  participating  in  test  runs. 

The  controllers  appeared  more  relaxed  during  this  simulation  than  in  previous  simulations,  and 
conflicts  were  resolved  with  relative  ease.  After  observing  this  simulation,  the  controller 
technical  observers  concluded  that  the  3000-ft,  2.5-degree  localizer  offset  procedure  could  be 
conducted  safely  and  efficiently  in  the  operational  environment  as  long  as  controllers  were 
trained  similarly  to  how  they  were  trained  in  this  simulation. 

1 1 .6.2  Pilot  Technical  Observer  Assessment 

Aircraft  breakout  performance  was  a  function  of  the  controller  and  pilot  performance.  The 
performance  of  the  pilots  was  improved  over  the  previous  3000-ft  offset  approach  simulation. 
This  improvement  was  probably  due  to  the  pilot  awareness  training,  the  new  controller 
phraseology,  the  improved  controller  training,  and  the  requirement  to  hand-fly  the  aircraft  during 
the  breakout  maneuver.  The  effectiveness  of  each  of  these  factors  was  not  quantified  in  this 
simulation,  but  the  importance  of  each  can  be  addressed. 

The  pilot  awareness  training  most  likely  contributed  to  a  reduction  of  long  response  times  by  a 
number  of  pilots.  After  the  training,  the  pilots  were  more  aware  of  the  dangers  of  not  reacting 
promptly.  The  new  phraseology,  beginning  with  "Traffic  Alert",  resulted  in  fewer 
communication  problems  by  reducing  the  number  of  clipped  call  sign  transmissions  and  by 
alerting  pilots  to  the  fact  that  a  breakout  was  imminent. 

Controller  training  resulted  in  the  controllers  using  one  transmission  for  the  evasion  instruction 
instead  of  many.  This  gave  pilots  more  time  during  a  high  workload  period  to  accomplish  duties 
associated  with  the  breakout  instead  of  answering  air  traffic  controllers  on  the  radio.  The  overall 
controller  performance  during  this  simulation  improved  over  the  previous  simulation  and 
contributed  greatly  to  the  aircraft  breakout  performance. 

The  hand-flown  breakout  requirement  was  effective  in  significantly  reducing  the  time-to-tum 
during  breakout  maneuvers  flown  in  the  B747-400  simulator.  This  improvement  will  vary 
between  various  types  of  glass  cockpit  aircraft  because  of  autopilot  system  design  differences. 
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There  was  almost  no  measurable  improvement  in  less  sophisticated  aircraft  such  as  the  B727, 
because  the  breakout  would  most  likely  be  flown  manually  even  without  the  requirement. 

11.6.3  Closest  Point  of  Approach  Distribution  Analyses 

Although  no  TCVs  occurred  during  the  simulation,  the  distribution  of  CPAs  was  examined  to 
provide  information  on  the  degree  of  separation  maintained  for  all  WCBs  involving  flight 
simulators.  Figure  5  shows  that  the  mean  separation  for  at-risk  blunders  was  2,358.91  ft,  with  a 
standard  deviation  of  842.06  ft.  The  range  of  CPAs  was  from  5 14.00  ft  to  4,467.00  ft.  CPAs 
were  also  sorted  by  simulator  type  and  examined.  Table  5  details  the  results. 


Figure  5.  Closest  point  of  approach  distribution  for  all  worst-case  blunders. 


Table  5.  Closet  Point  of  Approach  by  Simulator  Type 


SITE 

SIMULATOR 

TYPE 

MEAN  CPA  (ft) 

STANDARD 
DEVIATION  (ft) 

NUMBER  OF 
CASES 

AVIA 

B727 

2654.67 

905.04 

63 

DELTA 

MD90 

2272.85 

872.56 

80 

GAT 

C421 

2235.84 

695.77 

14 

NASA 

B747-400 

2273.03 

776.09 

76 

OKCITY 

B727 

2183.80 

680.84 

20 
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1 1 .6.4  Response  Time  Analyses 


1 1 ,6.4. 1  Controller  Response  Time  Analyses 

Controller  response  times  for  all  blunders  were  determined  (i.e.,  299  blunders;  response  times  not 
able  to  be  extracted  from  5  blunders).  The  times  were  calculated  from  the  time  of  the  alert  onset 
(i.e.,  the  change  in  color  of  the  predictor  line  and  data  block  from  green  to  yellow)  to  the  time  the 
controller  keyed  the  microphone  to  communicate  with  the  pilot  of  the  evading  aircraft.  Response 
times  ranged  from  -3  seconds  to  3  seconds  (with  one  outlier  response  time  of  10  seconds).  The 
mean  response  time  for  all  blunders  was  0.5  seconds  with  a  standard  deviation  of  1.1  seconds. 
Only  positive  response  times  collected  from  the  real-time  simulation  were  used  in  the  Monte 
Carlo  simulations.  Figure  6  depicts  the  complete  frequency  distribution  of  controller  response 
times,  which  were  independent  of  blunder  range  from  threshold. 


Figure  6.  Controller  response  times  for  all  blunders. 
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1 1. 6.4.2  Pilot/Aircraft  Response  Time  Analyses 


Pilot/aircraft  response  times  for  the  evaders  were  extracted  using  data  collected  at  the  flight 
simulator  sites.  The  pilot/aircraft  response  was  measured  as  elapsed  time  from  the  beginning  of 
the  evader  controller  transmission  to  a  particular  event  (e.g.,  attainment  of  3  degrees  of  roll, 
increase  in  engine  pressure  ratio).  This  analysis  focuses  on  the  time  to  attainment  of  3  degrees  of 
roll  because  the  promptness  of  the  turning  response  appears  to  be  the  most  critical  in  assuring 
separation  of  aircraft  during  a  blunder  scenario. 

NOTE:  During  post-simulation  analyses,  it  was  noted  that  the  pilot/aircraft  response  times 
observed  in  the  cockpit  simulator  videotapes  appeared  to  be  2  to  3  seconds  faster  than  the 
response  times  derived  from  the  data.  Initial  investigation  revealed  a  delay  in  the  data  (both  time 
and  position)  on  the  PRM  display,  most  likely  due  to  delays  in  the  network  and  the  simulated 
PRM.  While  further  investigation  is  necessary  to  confirm  the  source(s)  of  the  delays  and  to  find 
more  precise  figures  for  them,  one  can  consider  the  results  of  the  simulation  to  be  conservative. 
The  pilot/aircraft  responses  in  this  section  have  not  been  adjusted  for  the  delays,  but  could  be 
approximately  1.5  to  2.5  seconds  less  than  the  times  indicated  (0.5  seconds  was  subtracted  from 
2  to  3  seconds  because  the  specifications  of  the  production  system  include  a  maximum  delay  of 
0.5  seconds). 

Data  for  the  roll  response  times  (times  to  attainment  of  3  degrees  of  roll)  were  available  for  most 
of  the  evasions  with  the  exception  of  27  from  the  NASA  B747  site  and  5  from  the  Oklahoma 
City  B727  site.  Roll  response  times  were  also  not  available  for  five  GAT  evasions  because  those 
aircraft  were  already  in  a  roll  in  the  direction  of  the  evasion  maneuver  when  the  evader  controller 
began  transmitting. 


The  distribution  of  roll  response  times  for  the  simulation  are  shown  in  Figure  7. 


Figure  7.  Roll  response  times  -  all  flight  simulator  sites. 
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As  can  be  seen  in  the  figure,  nearly  60  percent  of  the  roll  response  times  were  less  than  eight 
seconds.  Further,  the  data  indicated  only  small  differences  among  the  flight  simulator  sites.  A 
breakdown  of  roll  response  time  statistics  by  simulator  site  is  shown  in  Table  6.  The  aggregate 
statistics  for  all  sites  is  also  in  Table  6. 

Table  6.  Roll  Response  Time  Statistics 


CAUTION:  Actual  response  times  could  be  approximately  1.5  to  2.5  seconds  shorter  than  response  times 
depicted  on  this  chart  because  of  simulation  system  delays. 


AVIA 

B727 

Delta 

MD90 

GAT 

NASA 

B747 

OKC 

B727 

All 

Sites 

Median  Time  to  3  Degrees  Roll 

7 

8 

6 

8 

7 

8 

Mean  Time  to  3  Degrees  Roll 

7 

9 

7 

9 

8 

8 

Minimum  Time  to  3  Degrees  Roll 

4 

4 

3 

4 

4 

3 

Maximum  Time  to  3  Degrees  Roll 

19 

19 

18 

32 

18 

32 

HI _ _ 

INumber  of  Breakouts 

72 

95  10 

62 

24  |  263  | 

Analysis  of  the  data  collected  at  the  simulator  sites  also  revealed  that,  for  21  evasions  at  the  Delta 
MD90  simulator  site,  pilots  elected  to  execute  the  breakout  using  the  autopilot,  contrary  to  their 
training  for  the  simulation.  A  breakout  was  considered  to  be  executed  on  the  autopilot  if  3 
degrees  of  roll  was  attained  before  the  autopilot  was  disengaged.  The  roll  response  time 
distribution  for  the  21  autopilot  breakouts  (Figure  8)  shows  slower  roll  response  times  than  in  the 
aggregate  distribution  in  Figure  7.  For  information  on  additional  aircraft  breakout  performance 
results,  see  Appendix  L. 


Figure  8.  Roll  response  times  for  autopilot  breakouts  (MD90). 
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1 1.6.5  Controller  Breakout  Instruction  Content  Analyses 


All  controller  breakout  instructions  made  during  the  simulation  were  studied  in  detail  for  content 
and  delivery.  Use  of  standard  phraseology  and  the  number  of  transmissions  required  to  complete 
breakout  instructions  were  two  components  of  the  messages  that  were  analyzed.  In  addition,  the 
number  of  descending  breakout  instructions  and  the  number  of  blocked  or  clipped 
communications  was  determined. 

1 1 .6.5.1  Use  of  Standard  Phraseology  and  Standard  Altitudes  and  Headings  for  Runways 

Controllers  were  briefed  on  the  standard  phraseology  to  use  in  the  event  of  a  blunder  (see  Section 
7.1.1),  and  had  an  opportunity  to  rehearse  the  phraseology  during  practice  runs.  During  actual 
test  runs,  most  controllers  used  the  standard  phraseology  in  the  initial  breakout  instructions, 
although  they  were  not  required  to  do  so.  Furthermore,  most  used  the  standard  altitudes  and 
headings  assigned  to  particular  localizer  courses  and  runways.  However,  variations  in 
phraseology  did  occur.  Table  7  shows  the  frequencies  and  subsequent  percentages  of  the  use  of 
standard  and  nonstandard  phraseology  over  the  course  of  the  simulation. 

Table  7.  Use  of  Standard  Breakout  Phraseology  and  Standard  Altitudes/Headings  for  Runways 


Standard  Altitudes/Headings  for 
3000-Ft  Configuration 
(in  1st  Transmission) 

Used 

Did  Not  Use 

Standard  Phraseology 
(in  first  transmission) 

Used 

263  (87.7  percent) 

22  (7.3  percent) 

Did  Not  Use 

11  (3.7  percent) 

4  (1.3  percent) 

During  87.7  percent  of  the  blunder  events,  controllers  used  the  standard  phraseology  and  the 
standard  altitudes  and  headings  assigned  to  particular  runways  in  the  initial  breakout 
transmission.  During  7.3  percent  of  the  blunders,  controllers  used  the  standard  phraseology,  but 
assigned  nonstandard  headings  and  altitudes.  During  3.7  percent  of  the  blunders,  controllers  did 
not  use  the  standard  phraseology  (e.g.,  forgot  to  say  "Traffic  Alert"  or  assigned  only  a  heading), 
but  did  use  the  standard  altitudes  and  headings  assigned  to  particular  runways.  Finally,  during 

I. 3  percent  of  the  blunders,  controllers  did  not  use  the  standard  phraseology  and  did  not  use  the 
standard  altitudes  and  headings. 

I I . 6.5.2  Multiple  Transmissions 

The  number  of  transmissions  made  by  the  controller  in  directing  an  aircraft  to  break  out  of  an 
approach  was  also  examined.  In  67.1  percent  of  the  blunder  events,  controllers  completed  the 
breakout  instructions  in  one  transmission.  In  the  remaining  32.9  percent  of  the  blunder  events, 
controllers  transmitted  two,  three,  and  four  times  (i.e.,  multiple  transmissions)  to  evading  aircraft. 
The  breakdown  by  number  of  transmissions  is  shown  in  Figure  9. 
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The  multiple  transmissions  were  categorized  by  the  contents  of  the  subsequent  transmissions. 
There  were  several  reasons  for  the  multiple  controller  transmissions.  First,  33  percent  were 
repeats  of  the  initial  breakout  instructions.  In  most  of  these  cases,  controllers  repeated  breakout 
instructions  because  pilots  of  the  evading  aircraft  did  not  verbally  respond  directly  following  the 
messages,  and  the  controllers  thought  their  messages  were  not  received.  In  other  cases, 
controllers  repeated  breakout  instructions  as  a  means  of  emphasizing  the  urgency  of  the  situation. 


Figure  9.  Number  of  transmissions  required  to  complete  breakout  instructions  for  all  blunders. 

A  further  32  percent  of  the  multiple  transmissions  were  to  offer  additional  information  to  the 
pilots.  This  information  included  phrases  such  as:  "Best  rate  of  climb/tum,"  "No  delay  in  your 
tum/climb,"  "Expedite  your  tum/climb,"  "Tighten  up  your  turn,"  "Traffic's  deviating  off  your 
left/right  off  the  other  localizer,"  and  "Traffic's  less  than  a  mile." 

Fifteen  percent  of  the  multiple  transmissions  were  for  changes  to  initial  instructions.  All  of  the 
changes  were  to  headings.  Controllers  first  issued  standard  breakout  headings  of  either  090 
degrees  or  270  degrees,  then  came  back  and  changed  them  to  greater  turns,  such  as  to  040  or  300 
degrees. 

Ten  percent  of  the  multiple  transmissions  were  for  combinations  of  types  of  information  (repeat, 
additional,  change,  and  split)  within  a  transmission,  as  follows: 

•  5.1  percent:  repeat  and  additional  information 

•  3.4  percent:  change  and  additional  information 

•  0.8  percent:  change  and  repeat 

•  0.8  percent:  change  and  split  (split  means  the  controller  only  gave  the  turn  in  the  initial 
instruction,  and  gave  the  altitude  in  one  of  the  following  transmissions) 

The  rest  of  the  multiple  transmissions  (9  percent)  consisted  of  questions  to  evading  aircraft  pilots 
who  offered  no  verbal  response  following  breakout  instructions  such  as,  "Delta  625?"  and 
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"American  799,  how  do  you  hear  me?"  The  transmissions  also  consisted  of  confirmatory 
statements,  such  as,  "That’s  correct,"  and  "That's  affirmative,”  and  appreciative  statements,  such 
as,  "Thank  you,"  and  "Excellent  turn.” 

1 1 .6.5.3  Descending  Breakout  Instructions 

No  descending  breakout  instructions  were  issued  by  controllers  during  the  simulation.  All 
controllers  were  briefed  on  the  standard  breakout  phraseology,  which  included  a  climb 
instruction.  However,  they  were  still  given  the  option  to  issue  instructions  as  they  deemed 
appropriate  (see  Section  3.2.3).  Nevertheless,  no  controllers  opted  to  descend  aircraft  off  the 
localizer  courses. 

1 1 .6.5.4  Blocked  and  Clipped  Breakout  Instructions 

A  total  of  418  breakout-related  ATC  transmissions  were  made  over  the  course  of  the  simulation. 
Of  those  transmissions,  13  were  either  blocked  or  clipped  (3.1  percent),  with  the  majority  being 
clipped. 

Six  of  the  blocked/clipped  transmissions  occurred  during  the  controller's  initial  breakout 
instruction.  In  three  of  these  cases,  the  phrase  "Traffic  Alert"  was  clipped.  However,  the  rest  of 
the  instructions  were  not  clipped,  and  therefore  pilots  were  able  to  respond  in  a  timely  manner. 

In  the  other  three  cases,  the  controllers  decided  to  repeat  the  instructions  because  they  did  not 
hear  pilot  readbacks  following  the  initial  transmissions. 

Seven  of  the  blocked/clipped  transmissions  occurred  during  the  second  ATC  transmissions 
(transmissions  following  the  initial  breakout  instructions).  The  effects  were  little  to  none 
because  the  instructions  in  all  cases  were  repeats  of  the  initial  instructions,  which  were 
transmitted  without  interference  the  first  time. 

1 1 .6.6  Questionnaire  Analyses 

1 1 .6.6. 1  Controller  Questionnaire  Analyses 

Controllers  were  administered  questionnaires  at  the  end  of  each  day  and  upon  completion  of  their 
participation  in  the  simulation.  They  were  not  informed  of  the  results  of  their  participation  or  of 
the  simulation  as  a  whole  prior  to  completing  the  questionnaires. 

1 1 .6.6. 1 . 1  Controller  End-of-Dav  Questionnaires 

Controllers  rated  stress  levels,  mental  effort,  and  activity  levels  experienced  throughout  an  entire 
day  on  the  End-of-Day  Questionnaire.  All  questions  were  answered  on  a  scale  from  1  (minimal) 
to  5  (intense).  Stress  was  defined  as  anxiety,  frustration,  and/or  pressure  experienced  in 
remaining  alert  when  working  with  the  PRM  system  and  while  re-establishing  separation  during 
a  blunder.  Mental  effort  was  defined  as  concentration,  attention,  and/or  decision  making 
involved  in  trying  to  maintain  vigilance  and  in  detecting  and  resolving  blunders.  Activity  was 
defined  as  frequency  of  actions  and  time  spent  on  tasks  due  to  the  occurrence  of  blunders  and 
traffic  density.  Forty-two  questionnaires  were  completed  and  the  results  are  reported  in  Table  8. 
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Table  8.  Summary  of  Controller  End-of-Day  Questionnaire  Responses 


Rate  the  level  of... 

1 

MINIMAL 

2 

3 

MODERATE 

4 

*  1 
INTENSE 

Stress  experienced  in 
resolving  blunders 

17  percent 

33  percent 

38  percent 

7  percent 

5  percent 

19  percent 

58  percent 

21  percent 

2  percent 

0  percent 

Mental  effort  required 
to  detect  blunders 

0  percent 

29  percent 

37  percent 

29  percent 

5  percent 

Mental  effort  required 
to  resolve  blunders 

14  percent 

43  percent 

31  percent 

5  percent 

7  percent 

10  percent 

38  percent 

45  percent 

7  percent 

0  percent 

Activity  required 
during  the  entire  day 

9  percent 

43  percent 

43  percent 

5  percent 

0  percent 

1 1 .6.6. 1 .2  Post-Simulation  Controller  Questionnaire 

Each  controller  was  given  a  Post-Simulation  Questionnaire  at  the  end  of  his  participation  in  the 
simulation.  Questions  addressed  the  feasibility  of  the  tested  procedure  in  the  operational 
environment  and  the  associated  strategies  that  were  developed  to  resolve  blunder  situations. 

The  results  indicated  that  all  of  the  controllers  (100  percent)  were  supportive  of  the  procedure  as 
tested.  They  believed  simultaneous  approaches  to  runways  spaced  3000  ft  apart  with  one 
localizer  offset  by  2.5  degrees  could  be  conducted  safely.  Additional  comments  addressed  the 
need  to  train  controllers  and  aircrews,  particularly  for  the  close  parallel  operation.  Several 
controllers  commented  positively  on  the  effectiveness  of  the  standard  breakout  phraseology  (i.e., 
Traffic  Alert,  aircraft  call  sign,  turn  left/right  heading  (degrees),  climb  and  maintain  [altitude]). 

Controllers’  suggestions  for  improvements  to  the  procedure  included  requiring  standard  breakout 
procedures,  slanting  the  radar  scopes,  and  incorporating  flashing  TAs  on  the  target  data  blocks. 
One  controller  commented  that  the  radio  equipment  "must  be  able  to  block  out  pilot  radio 
transmissions." 

Controllers  were  asked  to  assess  the  communications  workload  for  breakouts  and  NTZ  entries 
relative  to  current  operations.  Three  controllers  rated  the  workload  as  less  than  current 
operations.  Five  rated  it  equal  to,  two  rated  it  greater  than,  and  two  rated  it  much  greater  than 
current  operations.  One  controller  commented  that  it  took  more  time  to  use  the  phrase  Traffic 
Alert";  another  controller  stated  that  the  phraseology  was  better  than  current  required 
phraseology. 

Controllers  were  also  asked  if  they  had  developed  any  specific  control  strategies  for  the 
simulated  approach  operation.  Most  controllers  agreed  that  scanning  techniques  and  prioritizing 
viewing  areas  were  most  important.  The  areas  included  scanning  both  finals  and  being  alert  for 
the  aircraft  flying  side  by  side,  paying  particular  attention  to  aircraft  inside  the  outer  marker. 


35 


and/or  scanning  for  predictor  lines.  Other  strategies  mentioned  involved  inter-controller 
coordination  strategies,  such  as  listening  to  the  other  controller's  altitude  and  turn  assignments 
before- transmitting  and  alerting  the  other  controller  of  a  blunder,  especially  if  the  blunderer  was 
nonresponding. 

The  controllers  were  then  asked  if  the  strategies  they  developed  were  specific  to  the  tested 
operation.  Seven  controllers  said  yes  and  five  controllers  said  no.  Three  of  the  five  controllers 
who  said  no  stated  that  the  strategies  were  not  specific  because  they  use  them  in  the  field  now. 

Controllers  were  asked  to  rate  the  adequacy  of  the  briefing  information  and  training  they  received 
prior  to  the  actual  test  runs.  All  controllers  commented  that  the  briefing  information  was  helpful, 
but  the  practice  runs  and  the  experience  gained  through  the  practice  were  the  most  important  part 
of  the  training.  One  controller  thought  even  more  training  on  breakout  procedures  would  have 
been  helpful. 

1 1. 6.6.2  Pilot  Questionnaire  Analyses 

At  the  end  of  their  participation  in  the  simulation,  all  51  flight  simulator  pilots  completed  Flight 
Crew  Opinion  Surveys.  These  surveys  covered  the  importance  of  the  training  materials,  the 
information  page,  the  additions  to  the  approach  plates,  and  the  new  phraseology.  All  pilots  were 
encouraged  to  add  written  comments. 

1 1 .6.6.2. 1  Preferred  Method  of  Flying  Approaches 

The  pilots  were  asked  how  they  would  fly  a  simultaneous  close  parallel  approach.  This 
information  can  be  used  in  future  simulations  to  assign  a  realistic  ratio  of  hand-flown  flight 
director  and  coupled  autopilot  (A/P)  approaches.  The  B727  and  glass  cockpit  pilots  had  similar 
opinions.  Eighty-one  percent  preferred  a  coupled  autopilot  approach,  as  shown  in  Figure  10. 
Only  40  percent  of  the  GAT  pilots  preferred  a  coupled  autopilot  approach,  while  50  percent 
preferred  hand-flying  the  aircraft  during  the  approach. 

Based  on  the  information  presented  on  your  approach  plates  and 
information  page,  in  combination  with  the  simulated  weather,  how  would 
you  choose  to  fly  close  parallel  approaches? 

(GAT  pilots  not  included) 


47  Pilots  Responded 


Figure  10.  Preferred  method  of  flying  close  parallel  approaches. 
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1 1.6.6.2.2  Amount  of  Crew  Coordination  Required 

It  was  important  to  ascertain  how  the  pilots  felt  about  crew  coordination  for  the  establishment  of 
future  training  requirements.  Most  pilots  agreed,  as  shown  in  Figure  11,  that  more  crew 
coordination  was  required  for  simultaneous  approaches  than  for  normal  approaches. 


Figure  11.  Crew  coordination  for  close  parallel  approaches  as  compared  to  normal  approaches. 

1 1 ,6.6.2.3  Increased  Awareness  Through  the  Airport  Information  Page 

Many  changes  were  made  to  the  airport  information  page.  The  pilots  were  asked  questions 
regarding  how  these  changes  affected  their  awareness  of  adjacent  aircraft  and  of  close  parallel 
approach  procedures.  The  majority  of  pilots  thought  that  their  awareness  was  increased  in  both 
areas,  as  shown  in  Figures  12  and  13. 


Figure  12.  Airport  information:  Awareness  of  Figure  13.  Airport  information:  Awareness  of 
adjacent  aircraft.  procedures. 


The  Airport  Information  Page  increased  my 
awareness  of  possible  traffic  in  close  proximity 
on  the  adjacent  approach. 

(80%  Agree) 


Neutral  Disagree 


Strongly 

49  Pilots  Dlsa§ree 

,,  ,  ,  0% 

Responded 


The  Airport  Information  Page  increased  my 
awareness  of  close  parallel  approach 
procedures. 

(83%  Agree) 

Agree 
60% 


Strongly 

Agree 

23% 


Neutral  Dlsa®ree 

13%  4% 


49  Pilots 
Responded 


Strongly 

Disagree 

0% 
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1 1 .6.6.2.4  New  Air  Traffic  Control  Phraseology 


The  simulation  used  new  controller  phraseology  in  the  issuance  of  breakout  instructions  (refer  to 
Section  6.1.1).  It  was  important  to  learn  of  the  pilot's  reaction  to  this  new  phraseology  and  to 
determine  the  degree  to  which  the  pilots  thought  it  was  effective.  The  data  in  Figure  14  show 
that  seventy-six  percent  of  the  pilots  surveyed  thought  the  new  phraseology  was  more  effective 
than  the  previous  phraseology. 


The  new  ATC  phraseology  ("Traffic  Alert")  coupled  with  the  word 
"immediately"  encouraged  me  to  respond  more  quickly  to  the  breakout 
maneuver  than  I  would  have  if  only  the  word  "immediately"  was  used. 

(76%  Agree) 


Figure  14.  New  air  traffic  control  phraseology  in  relation  to  the  production  of  faster  pilot 
response  times. 


1 1 .6.6.2.5  Increased  Awareness  Through  Video 


The  video,  RDU  Precision  Runway  Monitor:  A  Pilots'  Approach,  was  considered  to  be  an 
important  part  of  the  pilot  training.  As  illustrated  in  Figure  15,  83  percent  of  the  pilots  agreed  to 
the  statement,  "The  video  increased  my  awareness  of  simultaneous  close  parallel  approach 
operations." 


The  video  increased  my  awareness  of  simultaneous  close  parallel 
approach  operations.  (83%  Agree) 


49  Pilots  Responded 


Strongly  Disagree 
0% 


Figure  15.  Video  in  relation  to  increased  operational  awareness. 


38 


1 1 .6.6.2. 6  Effect  of  Pilot  Bulletins  on  Performance 


Two  questions  addressed  the  effectiveness  of  the  training  bulletins.  The  first  question  asked  the 
pilots  whether  or  not  the  bulletins  provided  a  better  understanding  of  what  was  expected  of  them 
during  the  simultaneous  close  parallel  approaches.  The  data  in  Figure  16  show  that  77  percent  of 
the  pilots  agreed  the  bulletins  increased  their  understanding  of  the  procedure.  The  second 
question  asked  the  pilots  if  the  training  bulletins  helped  them  to  execute  ATC-directed  breakout 
maneuvers.  The  data  in  Figure  17  show  that  59  percent  of  the  pilots  agreed  that  the  training 
bulletins  aided  in  the  execution  of  breakouts. 


The  training  bulletins  (Pilot  Awareness  and  Procedure-if  applicable) 
increased  my  understanding  of  what  is  expected  of  me  during 
simultaneous  close  parallel  approaches.  (77%  Agree) 


The  training  bulletins  helped  me  to  execute  ATC-directed  breakouts. 
(59%  Agree) 


Figure  16.  Pilot  training  bulletin:  Understanding.  Figure  17.  Pilot  training  bulletin:  Execution. 
1 1.6.6.2.7  No  Descending  Breakout  Altitude 

For  this  simulation,  a  note  was  incorporated  into  the  approach  plate  that  indicated  the  altitude  on 
the  approach  below  which  no  descending  breakouts  would  be  issued  by  the  controller.  A 
question  was  asked  to  determine  whether  or  not  this  note  was  important  to  the  pilots.  Figure  18 
confirms  the  importance  of  this  item  and  illustrates  that  73  percent  of  the  pilots  agreed  that  they 
were  better  prepared  for  the  possible  breakout.  It  should  be  noted  that  there  were  no  descending 
breakouts  issued  during  this  simulation. 


The  'no  descending  breakout  altitude'  shown  on  the  approach  plate  helped 
me  better  prepare  for  the  possible  breakout  maneuver. 

(73%  Agree) 

Agree 
48% 


Strongly  Disagree 
0% 


Figure  18.  No  descending  breakout  altitude  on  approach  plate. 
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1 1.6.6.2.8  Rank  of  Importance 


The  last  survey  question  asked  the  pilots  to  rate  the  importance  of  items  used  during  the 
simulation.  The  significance  of  this  question  was  that  it  rated  the  importance  of  each  to  the  pilot. 
The  most  important  item  according  to  the  pilots  was  the  use  of  the  phraseology,  "Traffic  Alert". 
More  than  half  of  the  pilots  rated  this  item  as  a  10.  The  responses  to  this  question  are  depicted 
graphically  in  Figure  19. 


Rank  the  importance  (from  1  to  10)  of  each  of  the  following  items  to 
■  Median  flying  a  close  parallel  simultaneous  approach. 


PRM:  A  Information  Approach  Box  on  Bulletin  Procedure  "Traffic  Descending 
Pilot's  Page  Plates  Approach  Bulletin  Alert"  Breakout 

Approach  Plate  Phraseology  Altitude 

48  Pilots 

Responded  Items  Used  for  Pilot  Training  and  Awareness  During  October  3000*  Simulation 
Figure  19.  Rank  of  importance  of  items  to  flying  a  close  parallel  approach. 

12.  Summary 

In  the  previous  3000-ft,  2.5-degree  localizer  offset  simulation,  the  following  three  problem  areas 
were  identified  that  prevented  the  procedure  from  being  recommended: 

a.  Pilot/aircraft  response  times  and  performances  were  inadequate  to  meet  the  target  level  of 
safety  defined  in  the  test  criteria. 

b.  Blocked  and  clipped  communications  contributed  to  the  severity  of  many  conflicts. 

c.  Controller  breakout  phraseologies  and  procedures  varied  in  content  and  duration, 
sometimes  resulting  in  split  transmissions  and  confusion  in  the  cockpit. 

The  TWG  formed  investigative  groups  to  determine  ways  to  improve  the  three  problem  areas.  It 
was  recognized  that  procedure  changes  would  be  needed  before  the  3000-ft,  2.5-degree  localizer 
offset  operation  could  be  recommended.  The  results  of  the  pilot  training  and  controller  training 
groups'  efforts  were  enhanced  training  packages  designed  to  increase  user  awareness  and 
improve  performance  during  the  conduct  of  the  closely  spaced  approach  operation. 

As  a  result  of  the  communications  group's  efforts,  it  was  decided  that,  in  implementing  this  close 
parallel  configuration,  ABDs  shall  be  required  in  the  ground  radios.  In  the  event  ABDs  are  not 
available,  the  air  traffic  facility  will  have  two  communication  frequencies  available  for  each  close 
parallel  runway,  allowing  the  tower  and  monitor  controllers  to  simultaneously  broadcast  on  both 
frequencies  to  reduce  the  effect  of  blocked  communications. 


■  Mean  10 
8 

Rank  of  Importance:  6 
10  -  Very  Important  4 
1  -  Not  Important  2 
0 


B  I  I  I  I  I  I  I 


Video:  RDU  Airport  Separate  Notes  in  Awareness  Breakout  Use  of 


No 
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All  of  the  TWG’s  established  test  criteria  were  met  in  the  October  1995  3000-ft  simulation.  The 
real-time  TCV  rate  was  0.0  percent,  with  an  upper  confidence  limit  for  the  true  value  of  2.1 
percent.  Using  the  response  data  from  the  study,  the  BRM  and  ASAT  Monte  Carlo  simulations 
had  observed  TCV  rates  of  1.9  percent  and  0.11  percent,  respectively,  with  upper  confidence 
limits  of  2.0  percent  and  0.12  percent,  respectively.  These  results  were  considerably  less  than  the 
maximum  acceptable  TCV  rate  of  6.8  percent.  In  addition,  there  were  no  unintended  entries  into 
the  NTZ  by  flight  simulators,  and  only  22  NBOs  (1.3  percent)  caused  by  normal  flight  behavior. 
Both  results  were  considered  to  be  acceptable  frequencies  by  the  TWG.  Controller 
communications  workload  was  also  determined  to  be  acceptable. 

13.  Conclusions 


This  simulation  tested  the  procedure  for  simultaneous  ELS  approaches  to  two  parallel  runways 
spaced  3000  ft  apart  with  one  localizer  offset  by  2.5  degrees,  using  the  PRM  system  with  a  1.0- 
second  update  rate.  The  MPAP  TWG  evaluated  both  controller  and  pilot  effectiveness  at 
resolving  conflicts,  the  frequency  of  NTZ  entries  and  NBOs,  and  the  ability  of  the  system  to 
support  a  target  level  of  risk  of  no  more  than  one  fatal  accident  per  25,000,000  approaches. 
Based  upon  their  observations  and  evaluations,  the  TWG  recommends  this  procedure  as  tested 
for  approval  in  the  operational  environment. 
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Glossary 


At-Risk  Blunder  -  (As  defined  for  this  simulation)  A  blunder  in  which  two  aircraft  would  have 
come  within  500  ft  of  one  another  without  controller  intervention. 

Blunder  -  (As  defined  for  this  simulation)  An  unexpected  turn  by  an  aircraft  already  established 
on  the  localizer  toward  another  aircraft  on  an  adjacent  approach. 

Breakout  -  A  technique  used  to  direct  aircraft  out  of  the  approach  stream.  In  the  context  of  close 
parallel  operations,  a  breakout  is  used  to  direct  an  aircraft  away  from  a  deviating  aircraft  while 
simultaneous  operations  are  being  conducted. 

Close  Parallels  -  Two  parallel  runways  whose  extended  centerlines  are  separated  by  at  least  3,400 
feet,  but  less  than  4,300  feet,  and  having  a  Precision  Runway  Monitoring  (PRM)  system  that 
permits  simultaneous  independent  ILS  approaches.  (FAA,  1995a)  (Pending  change) 

Closest  Point  of  Approach  (CPA)  -  (As  defined  for  this  simulation)  The  smallest  slant  range 
distance  between  two  aircraft  involved  in  a  conflict.  The  distance  is  measured  from  the  center  of 
each  aircraft. 

Confidence  Interval  -  A  statistically  defined  range  of  values  of  the  population  mean,  any  one  of 
which  is  likely  to  be  represented  by  the  sample  means. 

Conflict  -  (As  defined  for  this  simulation)  An  event  in  which  two  or  more  aircraft  approach  each 
other  with  less  than  the  minimum  allowable  airspace  separation.  A  conflict  occurs  if  there  is  less 
than  1000  ft  vertical  or  3  nm  horizontal  distance  between  aircraft,  unless  the  aircraft  are 
established  on  ILS  approaches  and  separated  by  an  NTZ  during  simultaneous  ELS  approaches. 

Controller  Technical  Observer  -  An  individual  who  observes  a  monitor  controller  position  during 
each  simulation  run.  Duties  include  documenting  discrepancies  between  issued  control 
instructions  and  actual  aircraft  responses;  assisting  in  alerting  responsible  parties  to  correct  any 
problems  that  may  occur  during  the  test  (e.g.,  computer  failure,  stuck  microphone);  assisting 
controllers  in  preparation  of  reports;  and  documenting  their  evaluation  of  the  data  in  a  technical 
observer  assessment  at  the  end  of  the  simulation. 

CPA  prediction  tool  -  A  software  tool  used  by  the  simulation  test  director  that  presents  a  window 
of  aircraft  alignments  for  predicting  separation  between  aircraft. 

Final  Monitor  Aid  (FMA)  -  A  high  resolution  color  display  that  is  equipped  with  the  controller 
alert  system  hardware  and  software  used  in  the  PRM  system.  The  display  includes  alert 
algorithms  providing  the  target  predictors,  a  color  change  alert  when  a  target  penetrates  or  is 
predicted  to  penetrate  the  NTZ,  a  color  change  alert  if  the  aircraft  transponder  becomes 
inoperative,  synthesized  voice  alerts,  digital  mapping,  and  like  features  contained  in  the  PRM 
system  (FAA,  1995a;  FAR,  1995). 
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Final  Monitor  Controller  -  Air  Traffic  Control  Specialist  assigned  to  radar  monitor  the  flight 
paths  of  aircraft  during  simultaneous  parallel  and  simultaneous  close  parallel  ILS  approach 
operations.  Each  runway  is  assigned  a  final  monitor  controller  during  simultaneous  parallel  and 
simultaneous  close  parallel  ILS  approaches.  Final  monitor  controllers  shall  utilize  the  Precision 
Runway  Monitor  (PRM)  system  during  simultaneous  close  parallel  ILS  approaches.  (FAA, 

1995a). 

Flight  Technical  Error  (FTE)  -  (As  defined  for  this  simulation)  The  accuracy  with  which  the 
pilot  controls  the  aircraft  as  measured  by  the  actual  aircraft  position  with  respect  to  the  desired 
aircraft  position.  It  does  not  include  blunders. 

Glide  Slope  Intercept  Altitude  -  The  minimum  altitude  to  intercept  the  glide  slope/path  on  a 
precision  approach.  The  intersection  of  the  published  intercept  altitude  with  the  glide  slope/path, 
designated  on  Government  charts  by  the  lightning  bolt  symbol,  is  the  precision  Final  Approach 
Fix  (FAF);  however,  when  ATC  directs  a  lower  altitude,  the  resultant  lower  altitude  intercept 
position  is  then  the  FAF.  (FAA,  1995a;  FAR,  1995) 

Indicated  Airspeed  (IAS)  -  The  speed  shown  on  the  aircraft  airspeed  indicator.  This  is  the  speed 
used  in  pilot/controller  communications  under  the  general  terms,  airspeed.  (FAA,  1995a;  FAR, 
1995) 

Instrument  Landing  System  (ILS)  -  A  precision  instrument  approach  system  which  normally 
consists  of  the  following  electronic  components  and  visual  aids:  localizer,  glide  slope,  outer 
marker,  middle  marker,  and  approach  lights.  (FAA,  1995a;  FAR,  1995) 

Instrument  Meteorological  Conditions  GMC)  -  Meteorological  conditions  expressed  in  terms  of 
visibility,  distance  from  cloud,  and  ceiling  less  than  the  minima  specified  for  visual 
meteorological  conditions.  (FAA,  1995a;  FAR,  1995) 

Localizer  Offset  -  An  angular  offset  of  the  localizer  from  the  extended  runway  centerline  in  a 
direction  away  from  the  No  Transgression  Zone  (NTZ)  that  increases  the  Normal  Operating  Zone 
(NOZ)  width.  An  offset  requires  a  50  foot  increase  in  decision  height  (DH)  and  is  not  authorized 
for  CAT  H  and  CAT  m  approaches  (FAA,  1995a). 

Mean  -  The  arithmetic  average;  or  the  sum  of  measurements  divided  by  the  total  number  of 
measurements. 

Median  -  The  middle  value  when  measurements  are  arranged  in  order  of  magnitude;  the  value  at 
the  50th  percentile  of  a  set  of  measurements. 

Multiple  Parallel  Approach  Program  Technical  Work  Group  (MPAP  TWG)-  A  group  of  FAA 
employees  representing  several  different  offices  (e.g.,  Secondary  Surveillance  Product  Lead,  Office 
of  System  Capacity  and  Requirements)  that  assembles  to  make  recommendations  on  multiple 
parallel  approach  procedures. 
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National  Airspace  System  (NAS)  -  The  common  network  of  U.S.  airspace;  air  navigation 
facilities,  equipment  and  services,  airports  or  landing  areas,  aeronautical  charts,  information  and 
services;  rules,  regulations  and  procedures,  technical  information  and  manpower  and  material. 
Included  are  system  components  shared  jointly  with  the  military.  (FAA,  1995a;  FAR,  1995) 

No  Transgression  Zone  (NTZ)  -  A  2000  foot  wide  zone,  located  an  equal  distance  between 
parallel  runway  final  approach  courses,  in  which  flight  is  not  allowed  (FAA,  1995a). 

Normal  Operation  Zone  (NOZ)  -  The  operating  zone  within  which  aircraft  flight  remains  during 
normal  independent  simultaneous  parallel  ILS  approaches  (FAA,  1995a). 

Nuisance  Breakout  (NBO)  -  (As  defined  for  this  simulation)  An  event  in  which  an  aircraft  is 
broken  out  of  its  final  approach  for  reasons  other  than  a  blunder,  loss  of  longitudinal  separation, 
or  lost  beacon  signal  (i.e.,  aircraft  goes  into  coast). 

Outer  Marker  (OM)  -  A  marker  beacon  at  or  near  the  glide  slope  intercept  altitude  of  an  ILS 
approach.  It  is  keyed  to  transmit  two  dashes  per  second  on  a  400  Hz  tone,  which  is  received 
aurally  and  visually  by  compatible  airborne  equipment.  The  OM  is  normally  located  four  to 
seven  miles  from  the  runway  threshold  on  the  extended  centerline  of  the  runway.  (FAA,  1995a; 
FAR,  1995) 

Pilot  Technical  Observer  -  A  pilot  who  participates  in  the  simulation  as  a  flight  simulator  site 
coordinator.  The  pilot  technical  observer  evaluates  operational  aspects  of  the  simulation  at  the 
sites  and  is  the  conduit  to  the  William  J.  Hughes  Technical  Center  for  that  area/phase  of  the 
simulation. 

Precision  Runway  Monitor  (PRM1  System  -  A  system  that  provides  air  traffic  controllers  with 
high  precision  secondary  surveillance  data  for  aircraft  on  final  approach  to  closely  spaced  parallel 
runways.  High  resolution  color  monitoring  displays  (FMAs)  are  required  to  present  surveillance 
track  data  to  controllers  along  with  detailed  maps  depicting  approaches  and  the  no  transgression 
zone.  (FAA,  1995a;  FAR,  1995) 

Simulation  Pilot  Operator  fSPO)  -  A  person  who  operates  a  Target  Generation  Facility  (TGF) 
computer  workstation  and  controls  the  trajectory  of  TGF  aircraft  by  computer  input  messages. 
The  SPO  usually  communicates  via  voice  circuits  to  ATC  personnel  in  the  laboratory  that  is 
being  used  to  simulate  an  operational  facility. 

Simultaneous  TT.S  Approaches  -  An  approach  system  permitting  simultaneous  ILS/MLS 
approaches  to  airports  having  parallel  runways  separated  by  at  least  4,300  ft  between  centerlines. 
Integral  parts  of  the  total  system  are  ILS/MLS,  radar,  communications,  ATC  procedures,  and 
appropriate  airborne  equipment.  (FAA,  1995a;  FAR,  1995) 

Site  Coordinator  -  A  current  or  retired  airline  pilot  with  MPAP  real-time  simulation  and  flight 
simulator  experience  who  observes  aircrews  during  their  approaches.  Duties  include  briefing 
aircrews,  providing  pilots  with  flight  information,  documenting  approach  information,  and 
administering  questionnaires  to  the  pilots. 
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Target  Generation  Facility  (TGF)  -  An  advanced  simulation  system  designed  to  support  testing  of 
current  and  future  ATC  systems  at  the  William  J.  Hughes  Technical  Center.  The  TGF  is  capable 
of  modeling  a  logical  view  of  the  ATC  environment  (airspace  volume  including  geographic  data, 
weather  data,  navigation  aids,  radar  sensors,  airport  data,  and  air  routes)  as  well  as  simulating 
dynamic  data  associated  with  the  movement  and  control  of  aircraft  through  the  selected  airspace. 

Target  Generation  Facility  (TGF1  Aircraft  -  Targets  generated  by  the  TGF  at  the  William  J. 
Hughes  Technical  Center.  TGF  aircraft  are  used  to  provide  additional  traffic  and  to  initiate 
blunders. 

Test  Criterion  Violation  (TCV)  -  (As  defined  for  this  simulation)  An  event  that  occurs  when  the 
CPA  between  two  aircraft  after  the  initiation  of  a  blunder  is  less  than  500  ft. 

Test  Director  -  The  individual  responsible  for  cueing  blunder  initiation  through  the  use  of  the 
CPA  prediction  tool  and  by  assessing  the  blunder  scripts.  The  test  director  is  the  liaison  between 
the  William  J.  Hughes  Technical  Center  and  the  flight  simulator  sites  during  the  simulation. 

Total  Navigation  System  Error  (TNSE)  -  (As  defined  for  this  simulation)  The  difference 
between  the  actual  flight  path  of  the  aircraft  and  the  path  it  is  intending  to  fly.  It  is  caused  by 
FTE,  avionics  error,  ILS  signal  error,  and  weather. 

Worst-Case  Blunder  fWCB)  -  (As  defined  for  this  simulation)  A  blunder  in  which  the 
blundering  aircraft  turns  30  degrees  towards  an  adjacent  approach  course  and  does  not  respond  to 
controller  instructions  to  return  to  course. 
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Acronym  List 


ABD 

Anti-Blocking  Device 

ASAT 

Airspace  Simulation  and  Analysis  for  TERPS 

ATC 

Air  Traffic  Control 

ATIS 

Automatic  Terminal  Information  System 

BRM 

Blunder  Risk  Model 

CPA 

Closest  Point  of  Approach 

DH 

Decision  Height 

E-Scan 

Electronic  Scanning 

El 

External  Interface 

FAA 

Federal  Aviation  Administration 

FAF 

Final  Approach  Fix 

FMA 

Final  Monitor  Aid 

FTE 

Flight  Technical  Error 

GAT 

General  Aviation  Trainer 

IAS 

Indicated  Airspeed 

ILS 

Instrument  Landing  System 

IMC 

Instrument  Meteorological  Conditions 

IP 

Initial  Point 

MPAP 

Multiple  Parallel  Approach  Program 

MVA 

Minimum  Vectoring  Altitude 

NAS 

National  Airspace  System 

NBO 

Nuisance  Breakout 

NOZ 

Normal  Operating  Zone 

NTSB 

National  Transportation  Safety  Board 

NTZ 

No  Transgression  Zone 

OM 

Outer  Marker 

PF 

Pilot  Flying 

PNF 

Pilot  Not  Flying 

PRM 

Precision  Runway  Monitor 

rms 

Root  Mean  Square 

SPO 

Simulation  Pilot  Operator 

SPW 

Simulation  Pilot  Workstation 

TCV 

Test  Criterion  Violation 

TERPS 

Terminal  Instrument  Procedures 

TG 

Target  Generation 

TGF 

Target  Generation  Facility 

TNSE 

Total  Navigation  System  Error 

TRACON 

Terminal  Radar  Approach  Control 

TWG 

Technical  Work  Group 

WCB 

Worst-Case  Blunder 
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Appendix  A 

Multiple  Parallel  Approach  Program  Summary 


Multiple  Parallel  Approach  Program  Simulations 


Phase 

Dates 

Purpose 

Approach 

Runway 

Spacing 

Display 

Simulated 

Radar 

Other 

TWG  Recommendation 

I 

5/16- 

6/10/88 

DFW 

Quadruple 

SANDERS/ 

DEDS 

ASR-9 

4.8s 

Approved 

n 

Mj 

DFW 

Triple 

5000  & 
8800  ft 

SANDERS/ 

DEDS 

ASR-9 

4.8s 

Approved 

11/29- 

2/9/90 

DFW 

Dual  and 
Quadruple 

5000  & 
5800  ft 
8800  ft 

SANDERS/ 

DEDS 

ASR-9 

4.8s 

Approved 

IV.a 

4/24- 

5/3/90 

National 

Standards 

Dual  and 
Triple 

4300  ft 

ARTS  m 

ASR-9 

4.8s 

Dual  in  Progress 

Triple  Not  Approved 

IV.b 

9/17- 

9/28/90 

National 

Standards 

Triple 

5000  ft 

ARTS  III 

ASR-9 

4.8s 

Approved 

V.a.l 

5/15- 

5/24/91 

National 

Standards 

Dual  and 
Triple 

4300  ft 

FMA 

ASR-9 

4.8s 

Approved 

V.a.2 

9/24- 

10/4/91 

National 

Standards 

Triple 

4000  ft 

FMA 

ASR-9 

4.8s 

Study  Closed 

No  Decision  Rendered 

V.a.2.2 

7/27- 

8/14/92 

National 

Standards 

Dual  and 
Triple 

4000  ft 

FMA 

ASR-9 

4.8s 

Inconclusive 

V.b.l 

& 

V.b.2 

3/18- 

4/5/91 

National 

Standards 

Dual  and 
Triple 

3000  ft 

FMA 

E-Scan 

1.0s 

Not  Approved 

V.b.3 

9/16- 

9/23/91 

National 

Standards 

Dual 

3000  ft 

FMA 

E-Scan 

1.0s 

1 -Degree 
Localizer 
Offset 

Study  Closed 

See  June  '94 

V.c 

w 

National 

Standards 

Triple 

3400  ft 

FMA 

ModeS 

2.4s 

Study  Closed 

No  Decision  Rendered 

V.d 

3/2- 

3/13/92 

Human  Factors 
Study 

Triple 

3400  ft 

FMA 

E-Scan 

1.0s 

1  Mr  Radar 
Accuracy 

N/A 

n/a 

9/8- 

9/25/92 

Density 
Altitude  Study 

Triple  and 
Quadruple 

7600  ft 
5280  ft 
5348  ft 

ARTS  in 

ASR-9 

4.8s 

Field  Elevation 
5431ft 

Study  Closed 

No  Decision  Rendered 

n/a 

11/16- 

11/20/92 

11/30- 

12/17/92 

DIA 

Triple 

7600  ft 
5280  ft 

FDAs 

FMA 

ASR-9 

4.8s 

Field  Elevation 
5431  ft 

Approved 

n/a 

6/6- 

6/17/94 

National 

Standards 

Dual 

3000  ft 

FMA 

E-Scan 

1.0s 

1 -Degree 
Localizer 
Offset 

Not  Approved 

n/a 

7/11- 

7/22/94 

National 

Standards 

Dual 

3000  ft 

FMA 

E-Scan 

1.0s 

2.5-Degree 

Localizer 

Offset 

Not  Approved 

n/a 

8/14- 

8/25/95 

National 

Standards 

Triple 

4000  ft 
5300  ft 

FMA 

E-Scan 

1.0s 

Evaluation 

In  Progress 

n/a 

10/16- 

10/27/95 

National 

Standards 

m 

FMA 

E-Scan 

1.0s 

2.5-Degree 

Localizer 

Offset 

Approved 

Appendix  B 

Monte  Carlo  Simulation 


B.l  Background 

Real-time  simulations  of  ATC  operations  have  been  used  increasingly  in  the  evaluation  of  new 
ATC  procedures  and  airport  configurations.  In  real-time  simulations,  controller  and 
aircrew/aircraft  performance  can  be  measured  individually  and  combined  in  the  system 
performance  measures.  One  benefit  of  conducting  real-time  simulations,  with  human  operators, 
is  that  unexpected  effects  of  the  procedure  on  system  performance  can  be  identified.  A  limitation 
of  real-time  simulations  is  that  only  a  relatively  small  number  of  conditions  can  be  tested 
economically.  Thus,  the  data  collected  is  usually  a  very  small  subset  of  all  the  possible 
conditions. 

Because  of  the  small  subset  of  data  collected  in  real-time  simulations,  there  is  a  relatively  large 
margin  of  error  in  the  observed  measures.  Confidence  intervals  are  typically  established  to 
estimate  the  range  of  the  simulation  results.  Since  the  sample  sizes  are  relatively  small,  the 
confidence  intervals  are  rather  large.  If  the  real-time  simulation  results  are  used  in  the  evaluation 
of  a  tested  procedure,  it  is  possible  that  a  decision  to  accept  or  reject  a  procedure  could  be  in 
error. 

One  way  to  refine  and/or  expand  the  results  of  a  real-time  simulation  is  to  conduct  a  computer 
simulation,  commonly  called  a  Monte  Carlo  simulation.  This  technique  has  been  used 
successfully  in  the  Multiple  Parallel  Approach  Program  (MPAP)  and  the  PRM  Demonstration 
Program  to  assess  the  probability  of  mid-air  collisions  during  simultaneous  parallel  approach 
operations. 

The  Monte  Carlo  simulation  uses  the  actual  aircrew/aircraft  and  air  traffic  controller  performance 
data  collected  in  the  real-time  simulation  and  examines  over  100,000  worst-case  aircraft 
blunders.  This  large  number  of  blunders  results  in  a  very  small  confidence  interval  for  the 
estimated  TCV  rate. 

Two  different  Monte  Carlo  models  are  being  used  in  the  MPAP,  which  are  the  Blunder 
Resolution  Model  (BRM)  and  the  Airspace  Simulation  and  Analysis  for  TERPS  (ASAT)  Model. 
The  models  were  developed  for  analyses  of  controller,  aircrew/aircraft,  and  system  performance, 
and  are  described  in  the  following  sections. 

B.2  Monte  Carlo  Simulation  of  Blunders  During  Simultaneous  Approaches 

The  BRM  was  originally  developed  for  the  PRM  Demonstration  Program  that  ended  in  1991.  It 
was  designed  to  measure  the  risk  of  a  mid-air  collision  during  simultaneous  operations  to  dual 
parallel  approaches.  This  was  the  first  PRM  risk  assessment  model  to  use  data  collected  in  the 
field  or  from  experiments  for  each  of  the  model  components.  Results  from  the  BRM  were  used 
in  the  development  of  the  current  standards  for  simultaneous  approaches  to  closely  spaced 
parallel  runways. 
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The  AS  AT  model  is  a  newer  generation  of  Monte  Carlo  computer  simulation.  It  was  developed 
to  perform  complex  multiple  aircraft  simulations  in  the  study  of  obstacle  clearance  and  airspace 
requirements  for  new  standards  and  the  re-evaluation  of  existing  standards. 

B.2.1  Monte  Carlo  Blunder  Models 

The  BRM  and  the  AS  AT  models  are  similar  in  their  basic  operation.  The  similarities  are 
described  below  in  Section  B.2.1.1.  Unique  features  of  the  BRM  and  the  AS  AT  models  are 
described  in  Sections  B.2.1.2  and  B.2.1. 3,  respectively. 

B.2.1.1  Common  Features  of  the  BRM  and  AS  AT 

The  purpose  of  each  Monte  Carlo  simulation  was  to  estimate  the  probability  of  a  TCV  as  a  result 
of  one  aircraft  turning  unexpectedly  from  one  ELS  approach  toward  another  aircraft  on  an 
adjacent  ELS  approach.  The  blundering  aircraft  turned  to  a  course  30  degrees  from  its  approach 
course.  The  aircraft  on  the  parallel  approach,  called  the  evader,  continued  its  ELS  approach  until 
contacted  by  ATC. 

Figures  B-l  and  B-2  show  the  components  and  flow  of  the  simulations  for  the  BRM  and  ASAT, 
respectively.  The  distance  between  aircraft  at  a  given  time  was  affected  by  several  stochastic 
processes:  initial  position  of  the  aircraft  relative  to  the  other  aircraft,  system  delay,  controller 
response  time,  communication  delay,  and  endangered  pilot/aircraft  response.  The  values  for  each 
component  were  declared  at  the  beginning  of  each  scenario.  Simulation  components,  such  as 
runway  configuration  and  surveillance  performance,  remained  fixed  for  all  trials. 

The  timing  of  the  NTZ  entry  alert  was  determined  based  upon  the  configuration  of  the  blundering 
aircraft  at  the  start  of  the  blunder.  The  timing  was  affected  by  the  position  of  the  aircraft  relative 
to  the  NTZ,  initial  heading,  speed,  and  random  surveillance  errors.  The  aircraft  positions  were 
then  analyzed  using  the  PRM  alert  logic.  An  alert  was  generated  when  an  aircraft,  the  blundering 
aircraft  in  this  case,  was  either  inside  the  NTZ  or  projected  to  be  inside  the  NTZ  within  10 
seconds. 

During  each  trial,  the  blundering  aircraft  descended  along  the  approach  path  until  it  reached  the 
blunder  start  position.  The  approach  course  had  a  3-degree  glide  slope  and  aircraft  decreased 
speed  in  accordance  with  data  collected  in  the  real-time  simulation.  The  blundering  aircraft  then 
turned  toward  the  adjacent  approach  using  a  standard  rate  turn  of  3  degrees  per  second.  The 
aircraft  continued  turning  until  the  blunder  heading  was  reached.  The  blundering  aircraft  then 
continued  at  the  final  blunder  heading  until  the  end  of  the  trial.  The  blunders  were  executed 
between  1.0  nm  and  12.0  nm  from  the  runway  threshold. 

System  delay,  communication  delay,  and  controller  response  time  were  based  on  parameter 
assignments.  System  delay  is  defined  herein  as  the  interval  between  the  start  of  aircraft 
interrogation  to  the  time  the  detected  or  tracked  target  appears  on  the  controller  display.  System 
delay  was  a  constant  based  upon  the  PRM  system  specification.  It  was  set  to  0.46  second  for  the 
ASAT  and  0.50  second  for  the  BRM.  The  communication  delay  was  a  random  occurrence  and  as 
such  was  incorporated  into  the  pilot  response  times  generated  in  the  real-time  simulation. 
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Figure  B-l.  PRM  Blunder  Resolution  Model. 


Figure  B-2.  Airspace  Simulation  and  Analysis  for  TERPS  (ASAT)  Model. 
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Controller  response  times  were  randomly  selected  from  data  collected  in  the  real-time  simulation. 
Each  response  time  was  measured  as  elapsed  time  from  the  start  of  the  NTZ  alert  until  the 
controller  of  the  evading  aircraft  began  a  message  to  the  evading  aircraft. 

The  slant  distance  between  blundering  and  endangered  aircraft  centers  was  updated  at  least  once 
every  second.  The  trial  was  stopped  when  the  closest  point  of  approach  (CPA)  occurred.  The 
miss  distance  was  added  to  the  set  of  miss  distances.  If  a  TCV  occurred,  that  information  was 
added  to  a  file  containing  TCV  data.  The  computer  simulations  then  began  the  process  again  by 
selecting  the  positions  and  response  variables  for  the  next  trial. 

B.2.1.2  Unique  Features  of  the  BRM 

The  BRM  used  computer-generated  tracks  for  the  blundering  aircraft,  which  were  based  on  a 
spatial  flight  trajectory  model,  and  actual  tracks  recorded  by  cockpit  simulators  during  the  real¬ 
time  simulation  for  evading  aircraft  tracks.  The  evading  aircraft  track  was  randomly  selected 
from  a  set  of  track  files,  which  were  grouped  according  to  position  along  the  approach  course. 
This  ensured  that  the  performance  and  configuration  of  the  evading  aircraft  were  appropriate  for 
the  location  of  the  event.  The  range  groupings  were  as  small  as  possible  to  ensure  correct  speeds 
and  maneuvers,  while  providing  at  least  50  evader  responses  per  range.  For  each  trial,  the  evader 
track  was  shifted  slightly  along  the  approach  so  it  would  be  aligned  with  the  starting  position 
randomly  selected  at  the  start  of  the  trial.  The  track  was  also  aligned  in  time  so  that  the  start  of 
the  ATC  breakout  instruction  in  the  BRM  simulation  matched  the  point  where  it  occurred  during 
the  recording  of  the  track  in  the  real-time  simulation. 

The  benefit  of  this  method  was  that  it  accurately  portrayed  the  observed  pilot/aircraft  response 
characteristics.  There  were  no  assumptions  about  independence  of  actions,  the  possible 
combination  of  actions,  or  the  correctness  of  the  responses.  The  actual  sequencing  of  pilot 
actions  and  the  actual  inputs  to  the  aircraft  were  represented.  The  limitation  was  that  only  a 
small  subset  of  responses  were  represented.  Since  a  single  outlier  response  could  have  been 
over-represented  because  of  sample  size,  the  TCV  results  tended  to  be  conservative. 

In  the  BRM,  the  range  of  the  evading  aircraft  was  randomly  chosen  to  be  within  1.5  nm  of  the 
blundering  aircraft.  This  represented  the  independent  sequencing  of  aircraft  on  adjacent 
approaches  that  occurs  during  simultaneous  parallel  approaches.  A  factor  of  1/17  was  then 
applied  to  estimate  the  percentage  of  at-risk  blunders.  This  was  the  same  technique  that  was  used 
to  convert  the  0.4  percent  general-case  TCV  rate  criterion  from  the  PRM  Demonstration  Program 
to  the  6.8  percent  criterion  for  this  study.  This  technique  resulted  in  a  conservative  estimate  of 
the  TCV  rate  because  it  included  TCVs  that  might  not  have  been  associated  with  at-risk 
configurations.  Rather,  these  TCVs  were  the  result  of  human  actions.  The  benefit  was  that  these 
non  at-risk  TCVs  were  represented  in  the  results  and  could  be  identified  and  analyzed  for 
contributing  factors. 

B.2.1.3  Unique  Features  of  the  AS  AT  Model 

The  AS  AT  model  combined  a  high  fidelity  flight  dynamics  and  FMS/autopilot  model  with 
atmospheric  models  for  wind,  temperature,  and  pressure.  The  flight  dynamics  and  FMS/autopilot 
models  were  developed  from  flight  data  supplied  by  aircraft  manufacturers.  The  models  were 
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comparable  in  fidelity  to  the  flight  dynamics  models  used  in  certified  motion-based  flight 
simulators.  The  types  of  aircraft  modeled  using  the  ASAT  included  a  B747,  B727,  MD90, 
ATR42,  and  an  ATR72.  The  atmospheric  model  was  comparable  to  those  used  in  certified 
motion-based  flight  simulators.  The  ASAT  also  modeled  radar  tracking  systems  and  human 
performance. 

Evader  pilot  and  aircraft  response  characteristics  were  extracted  from  the  real-time  simulation 
data  to  produce  evader  flight  tracks  in  the  ASAT  model.  The  empirical  data  were  grouped 
according  to  similarities  of  the  data.  For  example,  the  maximum  bank  angles  for  the  B747  and 
MD90  were  found  to  be  similar,  so  their  maximum  bank  angle  data  were  grouped  together  to 
form  one  distribution.  The  maximum  bank  angles  for  the  B727s  at  AVTA  and  Oklahoma  City 
formed  separate  distributions  and  were  represented  separately  in  the  conduct  of  the  ASAT  model. 
Probability  density  distributions  of  aircraft  performance  data  were  developed  for  each  aircraft 
group.  These  data  included  indicated  airspeeds,  roll  rates,  maximum  bank  angles,  and  rates  of 
climb.  Since  the  ATR42  and  ATR72  were  not  simulated  during  the  real-time  simulation,  data  for 
those  aircraft  were  not  available.  Probability  curves  for  the  B747  were  used  to  provide  the 
evasion  maneuvers  of  the  ATR42  and  ATR72.  It  was  felt  that  these  distributions  would  provide 
conservative  evasion  maneuvers  for  the  two  ATR  aircraft. 

The  benefit  to  fitting  continuous  curves  to  the  pilot  and  aircraft  performance  characteristics  was 
that  a  complete  spectrum  of  responses  could  be  represented.  This  reduced  the  effect  of  a  single 
combination  of  performance  characteristics  on  the  TCV  rate.  In  the  implementation  of  ASAT  for 
this  simulation,  the  combinations  of  roll,  pitch,  and  engine  performance  were  assumed  to  be 
independent. 

In  the  ASAT,  the  aircraft  were  positioned  so  that  the  evading  aircraft  was  at  risk.  This  was 
accomplished  by  first  selecting  a  position  for  the  evading  aircraft.  The  position  of  the  evading 
aircraft  was  determined  laterally  and  vertically  by  random  sampling  from  the  probability 
distributions  found  in  the  ICAO  Collision  Risk  Model  (CRM).  The  longitudinal  position  of  the 
evading  aircraft  was  set  at  an  integral  number  of  nautical  miles  from  the  threshold  ranging  from  2 
to  12  nm.  The  position  of  the  blundering  aircraft  also  was  determined  laterally  and  vertically  by 
random  sampling  from  the  probability  distributions  found  in  the  ICAO  CRM.  The  blundering 
aircraft  was  positioned  longitudinally  by  first  positioning  it  somewhat  closer  to  the  threshold  than 
the  evading  aircraft.  Then  the  model  was  activated  so  that  the  blundering  aircraft  performed  the 
blunder  maneuver,  but  the  evading  aircraft  flew  on  course  without  an  evasion  maneuver. 

As  the  blunderer  passed  the  lateral  position  of  the  evader,  the  simulation  was  stopped  and  the 
miss  distance  was  used  to  adjust  the  starting  position  of  the  blundering  aircraft  so  that  the 
evading  aircraft  would  be  at  risk.  The  model  was  activated  again  and  the  evading  aircraft 
performed  an  evasion  maneuver.  The  roll  rate,  climb  rate,  maximum  bank  angle,  and  heading 
change  of  the  evading  aircraft  were  randomly  selected  from  distributions  derived  from  the  real¬ 
time  simulation.  These  were  categorized  based  upon  the  aircraft  positions  at  the  blunder  start. 
This  method  can  create  an  unlimited  combination  of  aircraft  tracks  and  aircraft  pairings. 

This  method  resulted  in  a  very  accurate  determination  of  TCVs  associated  with  at-risk  blunder 
configurations.  It  was  also  comparable  to  the  method  used  in  the  real-time  simulation. 
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B.2.2  Configuration  of  the  Monte  Carlo  Models 


For  this  study,  the  Monte  Carlo  simulations  were  configured  to  match,  as  closely  as  possible,  the 
real-time  simulation: 

a.  Same  runway  and  approach  configuration 

b.  Similar  TNSE  errors 

c.  Similar  aircraft  speed  distributions 

d.  Similar  distributions  of  blunder  starting  points 

e.  Same  blunder  configuration  (30-degrees,  nonresponding) 

f.  Similar  distributions  of  controller  and  endangered  aircraft  responses 

The  controller  response  times  relative  to  the  auditory  NTZ  entry  alert  were  extracted  from  the 
real-time  simulation.  The  controller  responses  were  analyzed  for  any  factors  that  could  have 
influenced  the  type  of  breakout  instruction  or  delay.  Analyses  indicated  that  the  relative  position 
between  the  blundering  and  evading  aircraft  did  not  affect  the  controller  response  time.  Analyses 
also  showed  that  the  distance  of  the  blunder  from  the  threshold  did  not  affect  the  controller 
response  time.  A  probability  density  curve  was  fit  to  the  data.  This  filled  in  the  gap  between  3  s 
and  10  s,  for  which  there  were  no  real-time  ATC  response  data,  resulting  in  a  continuous 
distribution  of  responses  for  the  computer  simulations. 

It  was  felt  that  some  of  the  observed  response  times  by  the  air  traffic  controllers  were  extremely 
short  and  that  these  short  responses  were  induced  by  the  simulation  environment.  Air  traffic 
controllers  would  probably  not  react  before  the  NTZ  alert  in  the  operational  environment. 
Therefore,  the  AS  AT  model  restricted  controller  reaction  times  to  greater  than  or  equal  to  0.2 
seconds  in  the  analysis  of  the  approach  operation. 

Since  standard  pilot  procedures  limit  the  aircraft  to  30  degrees  of  bank,  the  maximum  bank  angle 
in  the  AS  AT  simulation  was  limited  to  values  less  than  or  equal  to  30  degrees.  Similarly,  the 
maximum  roll  rate  was  limited  to  8  degrees  per  second,  and  the  maximum  rate  of  climb  was 
limited  to  4000  feet  per  minute.  Limiting  the  distributions  in  this  way  resulted  in  slower  evasive 
maneuvers  and  thus  a  more  conservative  simulation. 

B.2.3  Monte  Carlo  Output 

The  output  of  each  Monte  Carlo  simulation  was  a  distribution  of  CPAs  for  at  least  100,000  trials. 
To  estimate  the  TCV  rate,  the  number  of  CPAs  less  than  500  ft  was  divided  by  the  total  number 
of  at-risk  blunders.  The  TCV  rate  results  were  then  used  in  the  analyses  of  the  operational  safety 
of  the  procedure. 

Estimates  of  the  probability  of  a  blunder  occurring  during  actual  simultaneous  approach 
operations  can  be  compared  with  the  simulation  results  to  determine  the  safety  of  the  proposed 
operation.  The  TCV  rate  is  combined  with  factors  that  estimate  the  probability  of  aircraft 
alignment  (1/17),  the  probability  the  blundering  aircraft  does  not  respond  (1/100),  the  probability 
of  a  blunder  (1/2000),  and  the  probability  that  a  blunder  will  be  30  degrees  (1/100). 
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Additionally,  the  Monte  Carlo  simulations  can  be  used  to  conduct  sensitivity  analyses.  The 
sensitivity  analyses  can  be  used  to  examine  the  procedure  if  limits  are  set  on  system  performance. 
An  example  would  include  slowing  all  controller  responses  by  1  second. 

B.2.4  Conclusion 

The  Monte  Carlo  simulations  provide  a  means  of  enhancing  the  findings  of  a  real-time 
simulation.  They  produce  unlimited  combinations  of  data  from  the  limited  set  of  data  collected 
in  the  real-time  simulation.  They  provide  the  researcher  with  a  cost  effective  method  of  refining 
the  analyses  and  provide  more  accurate  results  to  compare  with  test  criteria.  By  refining  the 
results,  the  FAA  is  less  likely  to  incorrectly  accept  or  reject  a  simulated  air  traffic  control 
procedure. 
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Appendix  C 
Risk  Analysis 


C.l  Overview 

The  TWG  has  agreed  to  an  analysis  that  relates  the  collision  rate  observed  in  the  simulation  to 
the  test  criterion  of  less  than  1  fatal  accident  per  25  million  approaches  (4  x  10'8).  Analysis  has 
shown  that  if  the  collision  rate,  given  that  an  at-risk  Worst-Case  Blunder  (WCB)  has  occurred,  is 
less  than  6.8  percent,  then  the  overall  risk  of  a  collision  accident  resulting  from  the  simultaneous 
instrument  approach  procedure  will  be  less  than  the  test  criterion. 

MPAP  simulations  measure  the  Test  Criterion  Violation  (TCV)  rate,  (i.e.,  the  proportion  of  at- 
risk  WCBs  resulting  in  aircraft  proximity  of  less  than  500  ft).  If  one  assumes  that  a  TCV 
represents  a  collision,  then  the  TCV  rate  observed  in  the  simulation  is  an  estimate  of  the  risk  of  a 
collision  on  a  simultaneous  instrument  approach,  given  that  a  WCB  occurs  and  the  aircraft  are  at 
risk. 

The  real-time  simulation  TCV  rate  is  an  imprecise  estimate  of  the  collision  risk  resulting  from  a 
WCB  since  it  represents  only  a  small  sample  of  blunders  (usually  200  to  300  at-risk  WCBs  per 
simulation).  With  a  small  sample,  it  is  possible  for  the  measured  TCV  rate  to  exceed  6.8  percent, 
while  the  actual  collision  risk  is  less  than  6.8  percent,  and  vice  versa.  This  is  because  the 
sampling  error  is  large,  resulting  in  a  large  confidence  interval  for  the  true  collision  rate. 

Therefore,  two  Monte  Carlo  simulations  are  used  to  increase  the  sample  size  and  decrease  the 
error  in  the  measured  TCV  rate.  The  Monte  Carlo  simulations  use  human  and  aircraft  response 
data  from  the  real-time  simulation  to  generate  altogether  over  500,000  at-risk  WCBs.  The  Monte 
Carlo  sample  sizes  produce  refined  estimates  of  the  collision  risk.  After  validating  the 
consistency  of  the  Monte  Carlo  results  with  the  real-time  results,  the  Monte  Carlo  results  are 
used  to  determine  if  the  procedure  has  an  acceptably  low  level  of  risk. 

C.2  Risk  Analysis 

Several  events  must  occur  simultaneously  for  a  collision  to  occur  during  simultaneous  instrument 
approaches.  Clearly,  a  blunder  must  occur,  or  there  would  be  no  significant  deviation  from 
course.  Previous  testing  has  shown  that  blunders  other  than  WCBs  are  of  negligible  risk,  so  the 
blunder  must  be  a  WCB.  Also,  the  blundering  aircraft  must  have  a  critical  alignment  with  an 
aircraft  on  an  adjacent  course  (i.e.,  it  must  be  at  risk).  If  all  of  the  above  events  develop,  a  TCV 
will  occur  if  the  controller  and  pilots  cannot  react  in  sufficient  time  to  separate  the  blundering 
and  the  evading  aircraft.  In  addition,  one  collision  will  involve  two  aircraft  and  will  probably 
produce  two  accidents,  as  defined  by  the  National  Transportation  Safety  Board  (NTSB). 
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Assuming  that  a  TCV  will  result  in  a  collision,  the  probability  of  a  collision  accident  can  be 
expressed  in  mathematical  terms  by: 

(1)  P(Accident)  =  P(TCV  and  At-risk  and  WCB  and  Blunder)  x  2 

or 

(2)  P(Accident)  =  P(TCVIAt-risk  and  WCB  and  Blunder)  x 

P(At-risklWCB  and  Blunder)  x 

P(WCBIBlunder)  x 
P(Blunder)  x  2 


Where: 

P(TCV  and  At-risk  and  WCB  and  Blunder)  is  the  probability  of  all  relevant  events  occurring 
simultaneously  (i.e.,  an  at-risk  WCB  that  results  in  a  TCV). 

PtTCVI At-risk  and  WCB  and  Blunder)  is  the  probability  that  a  TCV  occurs  given  that  an  at- 
risk  WCB  has  occurred.  This  quantity  is  estimated  by  the  simulation  of  at-risk  WCBs  in 
the  real-time  and  Monte  Carlo  simulations  (i.e.,  the  TCV  rate  in  the  simulation). 

Pf  At-risklWCB  and  Blunder)  is  the  probability  that  a  WCB  has  critical  alignment  with  an 
aircraft  on  an  adjacent  approach.  Analysis  conducted  in  preparation  for  this  simulation 
indicates  that  a  value  of  1/17  is  a  good  approximation  of  this  quantity,  given  3  nm  in-trail 
spacing. 

PtWCBIBlunder)  is  the  probability  that  a  blunder  is  a  WCB.  This  probability  is  unknown, 
but  is  estimated  to  be  approximately  1/100  (PRM  Demonstration  Report,  1991). 

P (Blunder  )  is  the  probability  that  a  blunder  occurs  during  a  simultaneous  instrument 
approach.  This  rate  is  also  unknown,  but  is  estimated  to  be  no  more  than  1/2000 
approaches.  This  is  a  conservative  value  derived  from  the  risk  analysis  conducted  during 
the  PRM  demonstration  program.  Until  a  blunder  rate  estimate  can  be  derived  from  field 
data  of  actual  blunder  occurrences  or  other  evidence  suggests  using  a  different  value,  the 
TWG  has  agreed  to  use  1/2000. 

The  factor  of  2  represents  two  accidents  per  collision. 
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C.3  Target  Level  Of  Safety 


The  total  number  of  air  carrier  accidents,  as  well  as  the  number  of  fatal  accidents  on  final 
approach,  has  been  extracted  from  NTSB  data  for  the  time  period,  1983-1989.  This  number, 
together  with  the  total  number  of  ILS  approaches  flown  during  this  time  period,  leads  to  an 
estimated  fatal  accident  rate  during  ILS  operations  performed  during  IMC  of  4  x  10'7  fatal 
accidents  (ACC)  per  approach  (APP).  There  are  a  number  of  causes  of  accidents  during  final 
approach,  such  as  structural  failure,  engine  failure,  or  midair  collision.  An  initial  estimate  is  that 
there  are  nine  possible  causes  of  accidents  on  final  approach.  A  tenth  possible  accident  cause,  a 
collision  with  an  aircraft  on  an  adjacent  approach,  is  created  with  the  implementation  of 
simultaneous  parallel  approaches. 

For  simplicity  of  model  development,  it  is  assumed  that  the  risks  of  the  ten  potential  accident 
causes  are  equal.  Thus  the  contribution  of  any  one  of  the  accident  causes  would  be  one-tenth  of 
the  total  accident  rate.  Based  on  this,  the  target  safety  level  for  midair  collisions  on  simultaneous 
parallel  approaches  is  4  x  10'8,  or: 

1  ACC 
25  mill  APP 

C.4  Maximum  Allowable  Test  Criterion  Violation  Rate 

Since  the  only  undefined  variable  in  equation  (2),  used  to  compute  the  maximum  acceptable 
accident  rate,  is  the  TCV  rate,  it  is  possible  to  determine  the  maximum  allowable  TCV  rate 
which  would  meet  the  target  level  of  safety.  Knowledge  of  this  number  would  allow  the  TWG  to 
quickly  decide  if  the  simulated  operation  would  meet  the  target  level  of  safety.  The  maximum 
allowable  TCV  rate  may  be  found  from  following  analysis. 

Given  the  target  level  of  safety,  P( Accident)  =  4  x  10'8,  then  the  equation  (2)  becomes: 

P(TCVI At-risk  and  WCB  and  Blunder)  x  P(At-risklWCB  and  Blunder)  x  P(WCB Blunder) 
x  P(Blunder)  x  2  =  4  x  10'8 

or, 


(3) 

P(TCVIAt-risk  and  WCB  and  Blunder)  = 

4  x  10"8  x  1 

X 

1 

1  P(At-risk/WCB  and  Blunder) 

x  lxl 

P(Blunder)  2 

Substituting  values  from  (2)  into  (3): 

P  (WCB/B hinder) 

(4) 

P(TCVIAt-risk  and  WCB  and  Blunder)  = 

4  x  10'8  x  17  x  100  x  2000  x  1  =  6.8% 

1  1112 
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Thus,  if  the  simulation  results  support  the  assertion  that  the  probability  of  a  TCV,  given  that  an 
at-risk  WCB  occurs  (P(TCVIAt-risk  and  WCB  and  Blunder)),  is  less  than  6.8  percent,  then  the 
simultaneous  approach  procedure  simulated  should  have  an  acceptable  accident  rate. 

The  TCV  rate  derived  directly  from  the  real-time  simulation  is  subject  to  variation  due  to  the 
small  sample  of  at-risk,  worst-case  blunders  obtained  from  the  simulation.  To  compensate  for 
this  variation,  a  99-percent  confidence  interval  of  the  TCV  rate  is  computed  from  the  observed 
TCV  rate  and  sample  size.  If  the  largest  value  of  the  confidence  interval  is  less  than  6.8  percent, 
then  it  is  very  likely  (the  probability  is  0.995)  that  the  actual  TCV  rate  is  less  than  6.8  percent  and 
the  simulated  procedure  may  be  considered  to  have  an  acceptable  risk  of  collision.  However, 
given  the  small  sample  size,  it  is  possible  for  the  confidence  interval  to  extend  above  6.8%  even 
though  the  true  TCV  rate  is  less  than  6.8%. 

Two  Monte  Carlo  simulations  are  used,  based  on  data  derived  from  the  real-time  simulation,  to 
effectively  increase  the  sample  size  of  worst-case  blunders  to  over  500,000  in  all  and  provide  two 
more  estimates  of  the  TCV  rate.  Since  the  sample  sizes  are  very  large,  the  99-percent  confidence 
intervals  obtained  from  the  Monte  Carlo  simulations  will  be  much  smaller  than  the  confidence 
interval  obtained  from  the  real-time  simulation.  If  the  TCV  rates  observed  in  the  real-time 
simulation  and  the  Monte  Carlo  simulations  are  statistically  consistent  and  if  the  largest  values  of 
the  Monte  Carlo  simulation  confidence  intervals  are  less  than  6.8  percent,  then  the  simulated 
procedure  is  considered  to  have  an  acceptable  risk  of  collision. 

C  6  Summary  Of  The  Risk  Evaluation  Process 

The  process  used  by  the  TWG  to  determine  whether  the  simulated  operation  meets  the  target 
level  of  safety  may  be  summarized  as  follows: 

•  Conduct  the  real-time  simulation  and  determine  the  number  of  at-risk  WCBs  and  the 
number  of  TC Vs. 

•  Validate  the  observed  TCVs. 

•  Compute  the  TCV  rate  by  dividing  the  number  of  valid  TCVs  by  the  number  of  at-risk 
WCBs  in  the  simulation  (denoted  TCVrt). 

•  Calculate  a  99-percent  confidence  interval  for  TCVrt  using  standard  statistical  techniques 
(confidence  interval  for  the  proportion  parameter  of  a  binomial  distribution). 

•  Use  controller,  pilot,  and  aircraft  response  data  to  conduct  two  Monte  Carlo  simulations 
of  at  least  100,000  at-risk  WCBs,  as  discussed  in  Appendix  B.  Record  the  number  of 
TCVs  observed. 

•  Calculate  the  TCV  rates  for  the  Monte  Carlo  simulations  by  dividing  the  number  of  TCVs 
observed  in  the  Monte  Carlos  by  the  number  of  at-risk  WCBs  (denoted  TCVmc). 

•  Calculate  99-percent  confidence  intervals  for  TCVmc  using  standard  statistical  techniques 
(confidence  interval  for  the  proportion  parameter  of  a  binomial  distribution). 
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•  Compare  TCVrt  and  TCVmc  using  standard  statistical  techniques  (test  for  difference  of 
proportions)  to  ensure  that  the  Monte  Carlo  and  real-time  simulations  produce  consistent 
results. 

•  If  the  simulations  have  not  produced  consistent  results  (i.e.,  the  statistical  test  shows  a 
difference  of  proportions  with  99  percent  confidence  between  the  Monte  Carlo  and  real¬ 
time  simulations),  then  additional  analyses  must  be  conducted  to  reconcile  the  Monte 
Carlo  and  real-time  test  results. 

•  If  the  test  results  are  consistent  and  the  upper  limits  of  the  99-percent  confidence  intervals 
for  TCVmc  are  less  than  6.8  percent,  then  the  tested  operation  has  an  acceptable  risk  of 
collision. 


C-5 


Appendix  D 
Controller  Briefing 


CONTROLLER  BRIEFING 
FOR  THE 

MULTIPLE  PARALLEL  APPROACH  PROGRAM 

SIMULATION 


October  16  -  27, 1995 
FAA  Technical  Center 
Atlantic  City  International  Airport,  NJ 
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•  The  Multiple  Parallel  Approach  Program  (MPAP) 
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THE  MULTIPLE  PARALLEL  APPROACH  PROGRAM 
(MPAP)  TECHNICAL  WORK  GROUP  (TWG) 


•  Investigates  the  use  of  triple,  quadruple,  and  closely-spaced  dual  parallel 
runway  configurations  through  the  conduction  of  real-time  simulations. 

•  Comprised  of  Federal  Aviation  Administration  representatives  from  the 
Secondary  Surveillance  Product  Lead,  Office  of  System  Capacity  and 
Requirements,  Flight  Standards  Service,  Air  Traffic  Rules  and  Procedures 
Service,  Office  of  Air  Traffic  System  Management,  Air  Traffic  Plans  and 
Requirements  Service,  the  Southwest  Region,  and  the  FAA  Technical  Center. 


PROGRAM  GOALS 

•  To  increase  airport  capacity  by  conducting  multiple  simultaneous  parallel  ILS 
approaches,  including: 

-  triple  and  quadruple  parallel  runways. 

-  closely-spaced  dual  parallel  runways. 

•  To  examine  operational  issues  and  make  recommendations  for  the 
establishment  of  national  standards  for  multiple  simultaneous  parallel  ILS 
approaches. 
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DEFINITIONS 

(As  defined  for  this  simulation) 

Blunder  -  An  unexpected  turn  by  an  aircraft  already  established  on  the  localizer 
toward  another  aircraft  on  an  adjacent  approach. 

Breakout  -  A  technique  used  to  direct  aircraft  out  of  the  approach  stream.  In  the 
context  of  close  parallel  operations,  a  breakout  is  used  to  direct  an  aircraft 
away  from  a  deviating  aircraft  while  simultaneous  operations  are  being 
conducted. 

CPA  -  [Closest  Point  of  Approach]  The  smallest  slant  range  distance  between 
two  aircraft  involved  in  a  conflict.  The  distance  is  measured  from  the  center  of 
each  aircraft. 

NBO  -  [Nuisance  Breakout]  An  event  that  occurs  when  an  aircraft  is  broken  out 
of  its  final  approach  for  reasons  other  than  a  blunder,  loss  of  longitudinal 
separation,  or  lost  beacon  signal  (i.e.,  aircraft  goes  into  coast). 

NOZ  -  [Normal  Operating  Zone]  The  operating  zone  within  which  aircraft  flight 
remains  during  normal  independent  simultaneous  parallel  ILS  approaches. 

NTZ  -  [No  Transgression  Zone]  A  2000-ft  wide  zone  located  an  equal  distance 
between  parallel  runway  final  approach  courses,  in  which  flight  is  not  allowed. 

TNSE  -  [Total  Navigation  System  Error]  The  difference  between  the  actual  flight 
path  of  the  aircraft  and  its  intended  flight  path.  It  is  caused  by  flight  technical 
error,  avionics  error,  ILS  signal  error,  and  weather. 
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SIMULATION  PURPOSE 


•  To  evaluate  the  feasibility  of  and  make  recommendations  on  the  air  traffic 
control  systems  ability  to  support: 

Dual  simultaneous  ILS  approaches  to  two  parallel  runways  spaced 
3000  feet  apart  with  one  localizer  offset  to  the  left  by  2.5  degrees 
using  the  Precision  Runway  Monitor  (PRM)  system. 

•  To  evaluate  procedures  and  operational  issues,  NOT  individual  controller 
performance. 
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PRM  VIDEO 


DISCREPANCIES  FROM  VIDEO 


VIDEO: 

Parallel  Runway  Monitor 
Active  expansion 
Yellow  alert  zone 
Adjustable  deviation  lines 
3 -line  data  block 
Adjustable  predictor  line 
Adjustable  target  trails 
Caution,  Call  Sign 


SIMULATION: 
Precision  Runway  Monitor 
4:1  expansion  ratio 
No  Transgression  Zone  only 
200-ft  interval  deviation  lines 
2-line  data  block  w/  time  share 
10-second  predictor  lines 
5-second  target  trails 
Call  Sign  only 


D-6 


PRM  SYSTEM 


•  The  PRM  system  consists  of: 

-  Final  Monitor  Aid  display 

-  E-Scan  radar  system 

-  1.0-second  radar  update  rate 

•  The  FMA  display  is  a  high  resolution  20  x  20  inch  color  monitor  which 
displays  the  following  close  parallel  approach  information  at  an  expansion  ratio 
of  4:1: 

-  No  Transgression  Zone  (NTZ) 

-  Outlined  in  red 

-  2,000  feet  wide 

-  Equidistant  between  ILS  approach  courses 

-  Extends  1/2  mile  beyond  departure  ends  of  the  runways 

-  Deviation  lines 

-  Solid  white  lines 

-  200-ft  intervals  from  ILS  localizer  course 

-  Extended  runway  centerlines 

-  Dashed  white  lines 

-  Each  dash  and  each  space  equal  to  1  nm 

The  display  intensity  may  be  adjusted  as  desired.  All  other  variables  of  the 
display  must  remain  constant. 

.  An  aircraft  predictor  line  is  a  straight  line  projecting  from  the  aircraft  target. 
The  predictor  line  shows  where  the  aircraft  will  be  in  10  seconds,  if  the  aircraft 
continues  at  its  current  velocity  and  heading. 

•  When  an  aircraft  is  within  10  seconds  of  entering  the  NTZ,  the  aircraft  target 
and  data  block  changes  from  green  to  yellow.  Also  at  this  time,  an  auditory 
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alert  sounds,  giving  the  call  sign  of  the  deviating  aircraft.  When  an  aircraft 
enters  the  NTZ,  the  aircraft  target  and  data  block  changes  from  yellow  to  red. 
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AIRPORT  APPROACH  AREA  CONFIGURATION 


The  simulated  generic  airport,  referred  to  as  Charlie  Airport,  is  configured  as 
follows: 

•  Two  parallel  runways 

-  Spaced  3000  feet  apart 

-  10,000  feet  in  length 

-  Even  thresholds 

•  18L  localizer  offset  to  the  left  by  2.5  degrees 

•  Glide  slopes  of  3  degrees 

•  Outer  markers  located  5  nm  from  the  runway  thresholds 

•  Field  elevation  of  500  feet  MSL 

•  Minimum  vectoring  altitude  (MV A)  of  1800  feet  MSL 
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AIRPORT  CONFIGURATION 


Extended 

LOCALIZER  COURSE  Runway  LOCALIZER  COURSE 
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APPROACH  PLATE  18R 


TWGESSEN  CHARTS 

16  OCT  95 

lit! 

ATIS  Arrival 

xxx.x 

CHARLIE  Approach  (R) 

xxx.x 

CHARLIE  Tower  Rwys 

18R-36L 

127.2 

Ground  121.9 

18L-36R 

126.8 

CHARLIE,  USA 

CHARLIE  INTL 
1  ^  ^  ILS  Rwy  18R 

(CLOSE  PARALLEL) 

LOCI  08.5 

(NOTE:  NOBDENT) 


MSA 

HANSL  OM  AptElev  500 f 


11-0  MUST  BE  READ  PRIOR  TO  FLYING  THIS  CLOSE  PARALLEL  APPROACH 


No  descending  breakouts  will 
be  given  below  1800' 


HANSL ' 


IIS 


180°  108.5 

(NOTE:  NO  IPENT) 


} 


MM 


Simultaneous  CLOSE  PARALLEL 
approaches  authorized  with  ILS  Rwyl8L 
RADAR  and  GLIDE  SLOPE  REQUIRED.  I 


GS  741’  '(24V) 


J4ANSL _ 

GS  2143'  (1643) 


APT.  500'  o  06  50 

MISSED  APPROACH:  Climb  to  1000'  then  climbing  right  turn  to  5000'  via  heading  205°  and  expect 
radar  vectors 


rvr18ot1/2 


DAW 


STRAIGHT  -  IN  -  UNDING  RWY 

ILS 

7QQ\m>i 


18R 


TDZ  or  CL  out  ALS  out 


RVR  24  or  1/2 


RVR  40  or  3/4 


w 

Kfl 

besm 

in 

E9 

mm 

K&H 

mm 

mm 

|SM18M 

■ 

u 

■ 

u 

■ 

LOC  (GS  Out) 


Not  Authorized 


CIRCLE  -  TO  -  UND 


Not  Authorized 
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APPROACH  PLATE  18L 


TWGESSEN  CHARTS 


16  OCT  95 


(Tma) 


ATIS  Arrival 

XXX.X 

CHARLIE  Approach  (R) 

xxx.x 

CHARLIE  Tower  Rwys 

18L-36R 

126.8 

18R-36L 

127.2 

Ground  121.9 

CHARLIE,  USA 

CHARLIE  INTL 
ILS  Rwy  18L 
(CLOSE  PARALLEL) 

LOC  109.9 

'MSA  (NOTE:  NO  IDENT) 

GRETL  OM  Apt.  Elev 500 ' 


L  11-0  MUST  BE  READ  PRIOR  TO  FLYING  THIS  CLOSE  PARALLEL  APPROACH^  J 


No  descending  breakouts 
will  be  given  below  1800' 


Simultaneous  CLOSE  PARALLEL 
approaches  authorized  with  ILS  Rwy  1 8R. 
RADAR  and  GLIDE  SLOPE  REQUIRED. 


JAM 

GS  74V  (24V) 


_GRETL_ 

GS  2142'  (1642') 


TCH  55' 

TPZE-5QCL 


APT.  500'  o  06  5.o 

MISSED  APPROACH:  Climb  to  1000'  then  climbing  left  turn  to  4000'  via  heading  155°  and  expect 
radar  vectors. 


rvr18  « 1/2 


STRAIGHT  -  IN  -  LANDING  RWY 

OAfl-h  750*  ( 250 •) 


18L 


rvr  24  w  1  /2 


RVR  40  or  3/4 


— 

o 

mvxm 

warn 

kusb 

mm 

mznM 1 

■ 

■ 

■ 

Bl 

■ 

■ 

LOC  (GS  Out) 


Not  Authorized 


CIRCLE*  TO -LAND 


A 
B 

iZi  Not  Authorized 

C 
D 


D-12 


CONTROLLER  RESPONSIBILITY 


•  You  are  the  final  monitor  controller.  Your  responsibility  is  to  maintain 
longitudinal  and  lateral  separation  from  aircraft  on  the  adjacent  localizer 
between  the  simultaneous  approach  fixes  to  1/2  mile  beyond  the  departure  ends 
of  the  runways. 

•  The  aircraft  will  be  established  on  the  localizer  and  displayed  on  the  FMA. 

You  must  ensure  that  each  aircraft  has  established  contact  with  the  tower  prior 
to  the  first  simultaneous  ILS  approach  fix. 

.  OVERVIEW  OF  RESPONSIBILITY: 

All  turn-ons  and  final  approaches  are  monitored  by  radar.  Since  the  primary 
responsibility  for  navigation  rests  with  the  pilot,  instructions  from  the  controller 
are  limited  to  those  necessary  to  ensure  separation  between  aircraft.  Information 
and  instructions  are  issued,  as  necessary,  to  contain  the  aircraft  flight  path  within 
the  Normal  Operating  Zone  (NOZ).  Aircraft  which  are  observed  to  enter  the  No 
Transgression  Zone  (NTZ)  are  instructed  to  alter  course  left  or  right,  as 
appropriate,  to  return  to  the  desired  course.  Unless  altitude  separation  is  assured 
between  aircraft,  immediate  action  must  be  taken  by  the  controller  monitoring  the 
adjacent  parallel  approach  course  to  require  the  aircraft  in  potential  conflict  to 
alter  its  flight  path  to  avoid  the  deviating  aircraft. 

(FAA  Order  7210.3K,  p  12-45f) 

.  OVERRIDE  CAPABILITIES: 

Separate  monitor  controllers,  each  with  transmit/receive  and  override  capability  on 
the  local  control  frequency,  shall  ensure  aircraft  do  not  penetrate  the  depicted 
NTZ. 

(FAA  Order  7 110.65  J,  p  5-9-7b.6  note  1) 

.  NO  TRANSGRESSION  ZONE  (NTZ): 

The  primary  responsibility  for  navigation  on  the  final  approach  course  rests  with 
the  pilot.  Control  instructions  and  information  are  issued  only  to  ensure  that 
aircraft  do  not  penetrate  the  NTZ. 

(FAA  Order  7 1 10.65J,  p  5-9-7b.5) 
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Controller  Responsibility  (Continued) 

.  FINAL  MONITOR  WORK  PAD: 

Controllers  shall  use  a  pad  to  keep  track  of  aircraft  as  follows: 

1.  Determine  approach  sequence  from  data  block  and  write  down  aircraft  call 
sign. 

2.  Place  a  check  mark  beside  the  call  sign  when  the  aircraft  checks  on  tower 
frequency. 

3.  Draw  a  line  through  the  aircraft  call  sign  when  radar  monitoring  is  terminated 
for  that  aircraft. 

.  CONTROL  INSTRUCTIONS: 

The  aircraft  is  considered  the  center  of  the  digitized  target  for  that  aircraft  for  the 
purposes  of  ensuring  an  aircraft  does  not  penetrate  the  NTZ. 

1 .  Instruct  the  aircraft  to  return  immediately  to  the  correct  final  approach 
course  when  the  aircraft  is  observed  to  overshoot  the  turn-on  or  continue  on  a 
track  which  will  penetrate  the  NTZ. 

Phraseology: 

’YOU  HAVE  CROSSED  THE  FINAL  APPROACH  COURSE. 

TURN  (left/right)  IMMEDIATELY  AND  RETURN  TO 
LOCALIZER/AZIMUTH  COURSE." 

or 

"TURN  (left/right)  AND  RETURN  TO  THE  LOCALIZER/AZIMUTH 
COURSE." 

2.  Instruct  aircraft: 

a.  That  have  entered  the  NTZ  or  are  established  on  a  track  that  will  enter 
the  NTZ  to  alter  course. 

b.  On  the  adjacent  final  approach  course  to  avoid  a  deviating  aircraft. 
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Controller  Responsibility  (Continued) 

Phraseology: 

"TRAFFIC  ALERT,  (A/C  call  sign)  TURN  (left/right)  IMMEDIATELY 
HEADING  (degrees),  CLIMB  AND  MAINTAIN  (altitude)." 

(Proposed  change  to  FAA  Order  71 10.65) 

3.  Standard  breakout  instructions  for  the  simulation  are  as  follows: 

a.  Runway  18R:  TURN  RIGHT  IMMEDIATELY  HEADING  TWO 

SEVEN  ZERO,  CLIMB  AND  MAINTAIN  FIVE 
THOUSAND. 

b.  Runway  1 8L:  TURN  LEFT  IMMEDIATELY  HEADING  ZERO  NINE 

ZERO,  CLIMB  AND  MAINTAIN  FOUR  THOUSAND. 

4.  As  soon  as  feasible  exchange  traffic  and  wake  turbulence  information. 

.  SPEED  ADJUSTMENT: 

"Do  not  assign  speed  adjustment  to  aircraft  inside  the  final  approach  fix  on  final  or 
a  point  5  miles  from  the  runway,  whichever  is  closer  to  the  runway." 

(FAA  Order  71 10.65 J,  p  5-7-lb.4) 

PILOT  VIDEO 

COMMUNICATION 

•  NORDO  Aircraft 

•  Blocked  Communications 
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CONTROLLER  PARTICIPATION 


.  A  total  of  12  monitor  controllers  (6  each  week)  will  participate  throughout  the 
simulation. 

•  Three  two-hour  runs  will  be  conducted  each  day. 

•  Controllers  will  rotate  midway  through  each  run. 

•  Controllers  will  not  be  scheduled  to  participate  more  than  one  hour  per  two- 
hour  run,  and  will  not  participate  more  than  three  hours  a  day. 

.  Controller  assignments  will  be  equally  divided  with  respect  to  runways  and 
traffic  scenarios. 

•  Controllers  will  randomly  choose  letter  identification  codes  to  ensure 
anonymity.  The  codes  should  be  used  for  marking  questionnaires  and  referred 
to  when  reading  the  controller  schedule. 

•  Controllers  will  be  required  to  complete  a  short  questionnaire  at  the  end  of  each 
day  and  at  the  end  of  the  simulation. 
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CONTROLLER  SCHEDULE 

October  16-27, 1995 


DATE 

RUN 

18R 

18L 

10/16 

la  (Practice) 

A 

B 

Monday 

lb  (Practice) 

C 

D 

2a  (Practice) 

E 

F 

2b  (Practice) 

A 

C 

3a  (Practice) 

B 

E 

3b  (Practice) 

D 

F 

10/17 

4a 

E 

A 

Tuesday 

4b 

D 

B 

5a 

C 

F 

5b 

A 

D 

6a 

F 

C 

6b 

B 

E 

10/18 

7a 

F 

B 

Wednesday 

7b 

A 

E 

8a 

D 

C 

8b 

B 

A 

9a 

C 

D 

9b 

E 

F 
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CONTROLLER  SCHEDULE 

October  16-27, 1995 


DATE 

RUN 

18R 

18L 

10/19 

10a 

F 

A 

Thursday 

10b 

B 

C 

11a 

D 

E 

lib 

B 

A 

12a 

F 

E 

12b 

C 

D 

10/20 

13a 

A 

B 

Friday 

13b 

C 

D 

14a 

D 

F 

14b 

A 

C 

15a 

B 

E 

15b 

D 

F 
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CONTROLLER  SCHEDULE 

October  16-27, 1995 


DATE 

RUN 

18R 

18L 

10/23 

16a  (Practice) 

G 

H 

Monday 

16b  (Practice) 

I 

J 

17a  (Practice) 

K 

L 

17b  (Practice) 

G 

I 

18a  (Practice) 

H 

K 

18b  (Practice) 

J 

L 

10/24 

19a 

K 

G 

Tuesday 

19b 

J 

H 

20a 

I 

L 

20b 

G 

J 

21a 

L 

I 

21b 

H 

K 

10/25 

22a 

L 

H 

Wednesday 

22b 

G 

K 

23a 

J 

I 

23b 

H 

G 

24a 

I 

J 

24b 

K 

L 
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CONTROLLER  SCHEDULE 

October  16-27, 1995 


DATE 

RUN 

18R 

18L 

10/26 

25a 

L 

G 

Thursday 

25b 

H 

I 

26a 

J 

K 

26b 

H 

G 

27a 

L 

K 

27b 

I 

J 

10/27 

28a 

G 

H 

Friday 

28b 

I 

J 

29a 

K 

L 

29b 

G 

I 

30a 

H 

K 

30b 

J 

L 
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CONTROLLER  QUESTIONNAIRES 

.  You  will  be  required  to  complete  three  different  types  of  questionnaires  during 
and  following  the  simulation: 

Blunder  Statement 

Controller  Questionnaire 

Post-Simulation  Questionnaire 

•  You  will  receive  a  Blunder  Statement  from  the  technical  observer  monitoring 
your  position  following  each  run  in  which  two  aircraft  come  within  a  close 
proximity  of  one  another. 

•  You  will  receive  a  Controller  Questionnaire  from  the  designated  technical 
observer  at  the  end  of  each  day. 

•  You  will  receive  a  Post-Simulation  Questionnaire  from  the  designated  SRC 
representative  during  the  de-briefing  session  on  the  last  day  of  your 
participation  in  the  simulation. 

•  It  is  important  that  you  answer  the  questions  carefully  and  return  the  completed 
questionnaires  to  the  designated  SRC  representative  in  a  timely  manner. 
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FAA  TECHNICAL  CENTER  LAYOUT 


CONTROLLERS 
BREAK  ROOM 


SMOKING 

ROOM 


ACD-1  CONF.  ROOM 
(TWG) _ 


TATCA  LAB 
Visitors'  Viewing 
: DOOR :  Room 


PRM 

SIMULATION  LAB 


•  door : 


RED  BRICK 
BUILDING 


ENTRANCE 


STAIRS 


(FIRST  FLOOR) 


L  0  f 

LOBBY 

(FIRST  FLOOR) 


ENTRANCE 
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Appendix  E 
Pilot  Briefing 


PILOT  BRIEFING 


FAA  APPROACH  SIMULATION 


October  16-27, 1995 
FAA  Technical  Center 
Atlantic  City  International  Airport,  NJ 


INTRODUCTION 


•  Welcome  to  this  FAA  simulation.  Your  cooperation  in  this  project  is  greatly 
appreciated.  By  participating  in  this  simulation  you  will  assist  in  providing 
data  that  will  help  enhance  air  safety  and  increase  the  arrival  capacity  at  many 
airports  nationwide. 

•  In  this  simulation  you  will  be  flying  about  4  to  6 ILS  approaches  per  hour  to  a 
fictitious  airport  called  CHARLIE  INTERNATIONAL.  You  will  be  positioned 
at  a  point  located  about  20  NM  from  the  threshold  at  an  intercept  heading  of  20 
degrees  to  the  localizer.  You  will  be  using  the  simulator  headset  and 
microphones  through  which  you  will  hear  ATC  and  other  aircraft.  All  the 
necessary  approach  plates  will  be  provided. 

.  We  will  be  videotaping  and  sound  recording  the  entire  simulation.  Pilots  and 
their  airlines  will  not  be  identified.  Videotaping  has  proved  to  be  a  helpful  tool 
to  non  pilot  researchers  who  are  unfamiliar  with  cockpit  procedures  and  who 
do  not  realize  the  workload  involved  during  final  approach. 

•  We  will  request  your  social  security  number  and  pilot  license  number  for 
payroll  purposes  only.  You  will  receive  a  check  5  to  6  weeks  after  completing 
this  simulation.  If  you  do  not  receive  payment  within  6  weeks,  please  call  the 
SRC  Pilot  Group  at 

•  Thank  you  once  again  for  your  participation.  We  look  forward  to  working  with 
you. 
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FLIGHT  SIMULATOR  PROCEDURES 


•  The  site  coordinator  will  be  your  contact.  Refer  to  your  confirmation  letter  for 
his  name  and  phone  number. 

•  Before  each  approach  the  site  coordinator  will  provide  you  with  a  card 
containing  the  following  information: 

•  Call  sign  -  this  will  probably  be  another  airline  other  than  your  own. 

•  Transponder  code  -  it  is  very  important  to  squawk  the  proper  code  since 
this  is  used  in  exchanging  data  with  the  FAA  Technical  Center. 

•  Altitude  -  this  will  be  the  same  as  the  intercept  altitude  on  the  approach 
plates  provided  at  the  simulation  briefing. 

•  Airspeed  -  the  given  IAS  must  be  maintained  to  the  OM,  unless 
otherwise  directed  by  ATC. 

•  Aircraft  type  -  most  of  the  time  the  aircraft  on  the  card  will  be  different 
than  the  simulator  you  are  flying.  This  will  present  ATC  with  a  variety 
of  aircraft  on  their  display.  You  will  still  fly  the  simulator  as  you  would 
normally. 

•  ILS  Localizer  and  Tower  Frequencies  -  these  correspond  to 
frequencies  found  on  the  approach  plates  and  are  displayed  on  the  card 
for  your  convenience. 

•  ATIS  -  weather  will  remain  the  same  for  each  two  hour  period  and  the 
only  difference  in  the  three  ATIS  cards  will  be  the  wind. 

•  Please  use  the  simulator  pre-release  checklist  to  ensure  each  of  the  above 
items  is  set. 
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•  The  site  coordinator  will  specify  whether  the  approach  will  be  flown  using  a 
coupled  autopilot,  or  handflown  using  the  flight  director.  For  those  pilots 
flying  the  General  Aviation  simulator  you  may  be  requested  to  fly  a  raw  data 
approach. 

•  Pilots  will  be  provided  with  the  required  experimental  approach  plates  which 
will  vary  slightly  from  the  current  JEPPESEN  format.  These  plates  represent 
our  fictitious  airport  called  CHARLIE  INTERNATIONAL. 

•  You  will  be  released  from  position  freeze  on  a  20  degree  intercept  heading  to 
the  localizer  at  about  20  NM  from  the  threshold. 

•  When  released  you  must  assume  you  have  just  been  told  by  Approach  Control 
to  contact  Charlie  tower  when  established  on  the  localizer  and  about  17  miles 
out.  You  will  remain  on  the  same  frequency  during  the  entire  approach  phase. 

•  The  site  coordinator  will  tell  you  when  each  approach  exercise  ends. 

•  The  site  coordinator  will  assign  new  altitude,  heading,  airspeed  and 
transponder  code  after  the  approach  is  completed  so  that  you  can  setup  for  the 
next  approach. 
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PILOT  RESPONSIBILITIES 


You  will  be  using  call  signs  that  differ  from  your  airline,  so  it  is  very  important 
to  listen  carefully  for  the  correct  call  sign.  It  will  be  helpful  to  repeat  the  call 
sign  out  loud  before  starting  each  approach.  The  index  for  each  approach  will 
have  your  call  sign  printed  on  it.  Please  put  it  where  both  pilots  can  see  it. 

•  Maintain  a  sterile  cockpit  in  accordance  with  the  FARs  and  the  approach 
environment.  Remember  that  cockpit  conversation  will  be  recorded. 

•  If  paired  with  an  unfamiliar  crew  member  or  a  pilot  from  another  company, 
review  and  establish  individual  cockpit  duties  to  be  used  during  all  the 
approaches. 

•  Complete  the  approach  briefing  and  any  checklists  that  are  required  prior  to  the 
final  check  before  being  released  from  position  freeze. 

•  Review  the  simulator  pre-release  checklist  to  ensure  the  correct  parameters  are 
set  prior  to  leaving  the  Initial  Point  (IP). 

•  Do  not  discuss  this  simulation  with  other  crew  members  who  will  be 
participating. 

•  Answer  questions  from  the  site  coordinator  after  selected  approaches. 

•  Complete  the  Flightcrew  Opinion  Survey  at  the  end  of  your  participation. 
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QUESTIONNAIRE  ADMINISTRATION 


•  At  the  end  of  your  participation  in  the  simulation,  you  will  complete  a 
Flightcrew  Opinion  Survey.  Only  one  survey  should  be  completed  throughout 
your  participation.  Return  your  completed  survey  to  the  site  coordinator  before 
you  leave  the  simulator  area. 

•  This  is  a  unique  opportunity  for  today’s  line  pilot  to  participate  in  the  research 
and  the  development  of  ATC  procedures.  Any  comments,  recommendations, 
or  ideas  provided  from  your  direct  participation  will  be  used  to  form  a 
foundation  that  will  help  shape  the  future  of  the  National  Airspace  System. 
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•  SIMULATOR  PRE-RELEASE  CHECKLIST 

TRANSPONDER . SET 

INTERCEPT  HEADING . . DEG,  SET 

ALTITUDE . LEVEL _ FT,  SET 

AIRSPEED  TO  BE  MAINTAINED  UNTIL  OM . . KIAS,  SET 

LOCALIZER . . SET,  FOR  RWY _ 

TOWER  FREQ . . SET,  FOR  RWY _ 

APPROACH  MODE  (IF  APPLICABLE) . ARMED 

AUTOTHROTTLE  (IF  APPLICABLE) . ARMED 

CALL  SIGN . REVIEW 
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PILOT  BRIEFING  AND  TRAINING 


SIMULTANEOUS  CLOSE  PARALLEL  ILS  APPROACHES 

At  180  kts.  an  aircraft  that  has  entered  the  No  Transgression  Zone  can  cross  the  adjacent  parallel  course 
centerline  in  as  little  as  NINE  SECONDS.  Inattention,  or  failure  to  expeditiously  comply  with  a  final 
monitor  controller's  breakout  instructions  could  result  in  a  midair  collision.  Remember,  you  are  being 
broken  off  the  ILS  because  an  aircraft  from  the  adjacent  ILS  has  probably  deviated  off  course  and  is 
HEADING  YOUR  WAY.  When  pilots  hear  or  read  the  word  CLOSE  in  association  with  simultaneous 
parallel  approaches,  they  should  be  especially  aware  that  any  instructions  issued  by  a  controller  should 
be  immediately  followed. 

This  Flight  Operations  Bulletin  imparts  important  information  necessary  for  pilots  to  attain  the  increased 
level  of  pilot  awareness  required  for  safe,  efficient,  simultaneous  close  parallel  ILS  approach  operations. 
Important  pilot  questions  of  "why",  "how  will  I  know",  "what  can  I  expect",  and  "what  will  I  do"  are 
answered.  Hopefully,  "SIMULTANEOUS  CLOSE  PARALLEL  ILS  APPROACHES"  will  be  put  on 
your  list  of  important  aviation  terms  or  "buzzwords". 

WHY? 

Increased  crew  awareness  is  necessary  because  in  all  probability,  there  is  an  aircraft  operating  on  the 
adjacent  parallel  localizer  course  as  close  as  3400'  from  your  wingtip.  Failure  to  comply  with  ATC 
clearances,  tune  the  proper  localizer  frequency,  accurately  track  the  localizer  course  centerline,  or 
respond  to  controller  breakout  instructions  in  an  expeditious  manner  are  all  factors  that  may  lead  to  loss 
of  lateral  separation,  near-midair  collisions,  or  midair  collisions.  Attention  to  detail  is  mandatory! 

HOW  WILL  I  KNOW? 

Key  words  such  as  simultaneous  and  close  parallel  should  alert  pilots  of  the  need  to  increase  their 
awareness  level.  ATIS  will  broadcast  phraseology  such  as: 

Simultaneous  Close  Parallel  ILS  Runways  (number)  L/R/C  Approaches  in  use. 

For  a  new  approach  procedure  the  approach  plate  for  close  parallel  approaches  will  be  titled  CLOSE 
PARALLEL  ELS  RWY  (number)R.  For  existing  approach  procedures  a  separate  plate  will  be  issued. 
This  separate  plate  will  have  CLOSE  PARALLEL  in  parenthesis  under  the  approach  name.  The  plates 
will  also  have  the  note  SIMULTANEOUS  CLOSE  PARALLEL  APPROACHES  AUTHORIZED  WITH 
RUNWAYS  (number)  L/C/R  HAVING  (number  of  feet)  CENTERLINE  SEPARATION  WITH 
(number)  L/C/R,  and  GLEDESLOPE  REQUIRED. 

WHAT  CAN  I  EXPECT? 

If  you  or  the  aircraft  on  the  adjacent  localizer  course  fail  to  track  the  localizer  course  centerline  (or  worse 
yet,  track  the  wrong  localizer  course!),  final  monitor  controllers  with  frequency  override  capabilities  will 
issue  instructions.  Each  runway  has  a  dedicated  frequency  therefore  you  will  not  hear  final  monitor 
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radio  transmissions  to  aircraft  on  the  adjacent  localizer  course.  The  two  scenarios  requiring  final 
monitor  intervention  are  aircraft  deviations  from  the  localizer  course  centerline,  and  penetration  of  the 
No  Transgression  Zone  by  the  aircraft  on  the  adjacent  parallel  localizer  course.  For  aircraft  observed 
deviating  from  the  localizer  course  centerline  or  observed  to  overshoot  the  tum-on,  final  monitor 
controllers  will  use  phraseology  such  as: 

"TURN  (left/right)  AND  RETURN  TO  THE  LOCALIZER  COURSE"  or  "YOU  HAVE 
CROSSED  THE  FINAL  APPROACH  COURSE.  TURN  (left/right)  IMMEDIATELY  AND 
RETURN  TO  THE  LOCALIZER  COURSE". 

When  an  aircraft  fails  to  respond  to  final  monitor  controller  instructions  or  is  observed  penetrating  the 
No  Transgression  Zone,  the  aircraft  on  the  adjacent  parallel  localizer  course(s)  will  be  issued  breakout 
instructions  such  as: 

"(A/C  callsign)  TRAFFIC  ALERT  (A/C  callsign)  TURN  (left/right)  IMMEDIATELY  HEADING 
(degrees)  CLIMB/DESCEND  AND  MAINTAIN  (altitude)". 

A  descending  breakout  is  contrary  to  what  pilots  would  normally  expect  but  the  pilot  should  be  aware 
that  ATC  will  use  it  when  necessary.  If  the  final  monitor  controller  issues  a  descending  breakout,  the 
descent  will  not  take  the  pilot  below  the  Minimum  Vectoring  Altitude  (MV A). 

WHAT  WILL  I  DO? 

To  ensure  proper  preparation  for  "what  will  I  do"  the  approach  briefing  shall  address  the  possibility  of  an 
ATC  directed  breakout.  The  briefing  should  also  include  how  that  breakout  will  be  accomplished. 
Pilots  must  comply  immediately  with  all  final  monitor  controller  instructions.  Having  been  cleared  for 
the  approach  pilots  are  in  a  land  the  aircraft  mode.  For  this  reason  it  feels  unnatural  to  be  broken  off  the 
approach  particularly  if  you  have  the  localizer  and  glideslope  wired.  For  these  reasons,  pilots  can  be 
tempted  to  question,  or  hesitate  in  complying  with  the  final  monitor  controller’s  breakout  instructions. 

During  close  parallel  approach  operations,  immediate  execution  of  the  final  monitor  controller's 
instructions  is  mandatory.  Remember,  you  are  being  broken  off  the  approach  because  the  aircraft 
assigned  to  the  adjacent  parallel  localizer  course  has  failed  to  respond  to  the  final  monitor  controller's 
instructions  or  has  entered  the  No  Transgression  Zone-the  aircraft  is  heading  your  way.  There  is  simply 
not  enough  real  estate  between  you  and  the  deviating  aircraft  to  execute  the  breakout  in  a  leisurely 
manner.  For  example,  at  140  kts  ground  speed  an  aircraft  is  traveling  236’  per  second! 

Pilots  must  be  knowledgeable  of  aircraft  autoflight  systems  as  well  as  procedures  and  limitations.  This 
is  necessary  to  avoid  slow,  or  improper  aircraft  response  to  an  issued  clearance.  Crew  coordination 
items  must  be  thoroughly  understood  and  briefed  prior  to  commencement  of  the  approach.  This  is 
particularly  critical  for  automated  cockpits.  Remember,  during  an  ATC-directed  breakout,  you  can  be 
given  any  combination  of  turn  and/or  climb/descent  instructions. 
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PILOT  AWARENESS 


OPEN  BOOK  TEST 

Pilot  Awareness  Traming  for  Simultaneous  Close  Simdtaneous  Parallel  ILS  Approaches. 

1 .  Close  Parallel  in  describing  a  simultaneous  ILS  approach  means: 

A.  Runway  centerlines  are  less  than  4,300’  apart. 

B.  Runway  centerlines  might  be  only  3,400’  apart. 

C.  There  might  be  someone  making  an  approach  to  the  adjacent  runway  who  is  very,  very 
close. 

D.  All  of  the  above. 

2.  If  a  pilot  is  flying  a  simultaneous  close  parallel  ELS  approach  and  the  controller  tells  him  to  turn 
off  the  localizer  the  pilot  should: 

A.  Take  his  time  because  the  passengers  don’t  like  sudden  maneuvers. 

B.  Move  the  aircraft  as  quickly  as  practical  to  avoid  a  potential  mid-air  collision. 

C.  Not  turn  off  the  localizer,  because  the  instruments  read  on  course  and  you've  been 
cleared  for  the  approach. 

3.  Can  a  controller  give  a  pilot  a  descending  turn  off  the  localizer  when  the  pilot  is  on  an  ILS 
approach? 

A.  No,  not  if  the  aircraft  has  captured  the  localizer  and  glideslope. 

B.  No,  all  turns  off  the  localizer  must  be  accompanied  by  a  climb. 

C.  Yes,  provided  the  aircraft  is  not  descended  lower  than  the  minimum  vectoring  altitude 
(MV A).  If  that  is  what  it  takes  to  avoid  a  collision,  the  controller  will  direct  a  descending 
turn  off  the  localizer. 
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Answers: 


1. D 

2.  B 

3.  C 
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PAGE  2 


OPEN  BOOK  TEST 

4.  What  are  the  most  important  things  to  remember  about  simultaneous  close  parallel  ILS 

approaches? 

A.  Don’t  make  any  abrupt  turns  because  of  passenger  comfort,  and  always  question  every 
turn  off  the  ILS  localizer  given  by  ATC. 

B.  If  you  don’t  turn  immediately  off  the  localizer  when  directed  by  ATC,  perhaps  maybe 
he’ll  forget  about  you  and  you  can  get  in  on  time.  If  the  controller  is  being  unreasonable 
by  making  you  late,  stand  your  ground  and  don’t  let  him  intimidate  you,  after  all  you’re 
an  airline  captain. 

C.  There  is  probably  someone  very  close  along  side  of  you  making  an  approach  to  the  other 
runway.  If  ATC  tells  you  to  turn  off  the  localizer  it  means  that  the  airplane  along  side  of 
you  is  now  heading  your  way  and  it  might  hit  you  unless  you  move  the  airplane  quickly. 

5.  When  you  hear  ATC  transmit  TRAFFIC  ALERT,  what  kind  of  message  is  going  to  follow? 

A.  A  turn  off  the  ILS  for  someone  because  an  aircraft  on  the  parallel  runway  is  heading  his 
way. 

B.  There  is  a  new  ATIS  coming  up  and  the  controller  wants  everyone  to  listen  to  it. 

C.  The  highway  leading  into  the  airport  is  really  jammed  up  with  cars. 

6.  How  should  the  briefing  for  simultaneous  close  parallel  ILS  approaches  be  conducted? 

A.  No  briefing  is  necessary. 

B.  Use  the  standard  briefing  for  ILS  approaches. 

C.  Use  the  standard  briefing  for  ILS  approaches.  Additionally  brief  for  the  close  aspect  of 
the  approach,  the  possibility  of  an  ATC  directed  breakout  and  how  it  should  be 
conducted. 
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Answers: 


4.  C 

5.  A 

6.  C 
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***This  procedure  is  for  simulation  purposes  only*** 

Breakout  Procedure  Bulletin 


General  Discussion: 

Closely  spaced  (less  than  4300  feet  between  parallel  runway  centerlines)  ILS 
simultaneous  approaches  have  created  the  need  for  a  breakout  procedure.  Breakout  is  defined  as 
an  ATC  directed  departure  from  an  ELS  approach  prior  to  reaching  the  D/H.  Before  the  advent  of 
closely  spaced  ILS  simultaneous  approaches,  ATC  rarely  diverted  an  aircraft  from  an  ELS 
approach.  If  a  breakout  was  initiated  by  ATC,  it  was  usually  the  result  of  a  spacing  problem  and 
not  a  potential  collision  problem  with  another  aircraft.  It  is  forecast  that  closely  spaced  ILS 
simultaneous  approaches  will  increase  the  frequency  of  breakouts  and  the  spacing  between  the 
parallel  localizers  dictates  that  a  procedure  be  implemented  to  reduce  the  maneuver  times  of  the 
evading  aircraft. 

A  breakout  to  avoid  a  collision  is  considered  to  be  an  emergency-like  maneuver  and 
extraordinary  steps  in  the  breakout  procedure  are  needed.  Although  autopilot  use  is  encouraged 
for  a  closely  spaced  ILS  simultaneous  approach,  the  breakout  procedure  will  be  hand  flown.  It  is 
very  important  that  the  breakout  transmission  from  ATC  be  followed  immediately  and  the  only 
way  this  can  be  accomplished  quickly  is  by  disconnecting  the  autopilot  and  hand  flying  the 
airplane  through  the  maneuver.  The  pilot  must  keep  in  mind  that  a  descent  might  be  one  of  the 
options  that  the  controller  might  use,  providing  the  aircraft  is  above  the  minimum  vectoring 
altitude  (MV A).  The  pilot  can  count  on  the  MVA  not  being  below  1,000’  AGL  and  in  all 
probability  the  MVA  will  be  considerably  higher,  because  the  MVA  provides  at  least  1,000’ 
clearance  above  obstacles. 

Studies  have  found  that  using  an  autopilot  for  a  breakout  results  in  longer  breakout  times 
than  when  the  aircraft  is  handflown  through  the  maneuver.  Some  aircraft  take  longer  than  others 
when  using  the  autopilot,  but  for  standardization  purposes,  the  policy  will  be  to  make  all  ATC 
directed  breakouts  without  the  autopilot  connected. 


E-14 


***Thi$  procedure  is  for  simulation  purposes  only* 


Breakout  Procedure  Bulletin 


The  following  procedure  is  to  be  used  when  conducting  an  ATC  directed  breakout  from  a  closely 
spaced  ILS  simultaneous  approach: 

PILOT  FLYING  (PF): 


•  Disconnect  autopilot  and  point  the  aircraft  in  the  direction  the  controller  has 
directed  (including  climb  or  descent). 

•  Monitor  speed  and  consider  disconnecting  autothrottles  (if  applicable)  or 
overriding  with  manual  throttle  inputs. 

•  Command  to  clean  up  aircraft  using  normal  procedures. 

PILOT  NOT  FLYING  (PNF)  -  catch  up  the  flight  director  to  the  aircraft  flight  path: 

•  Turn  off  both  flight  directors 

•  Turn  on  PNFs  flight  director 

•  Set  new  heading. 

•  Set  new  altitude 

•  Make  other  inputs  as  necessary  to  match  the  flight  director  with  the  desired  flight 
path. 

•  When  PNFs  flight  director  matches  the  desired  aircraft  flight  path,  turn  on  PFs 
flight  director. 

NOTE:  It  is  important  that  the  PNF  turn  off  the  flight  director  of  the  PF,  so  that  the  PF  does  not 
have  to  fly  through  the  flight  director.  It  is  not  desirable  to  have  the  flight  director  telling  the 
pilot  to  do  the  opposite  of  what  ATC  wants  the  pilot  to  do. 

THIS  BREAKOUT  PROCEDURE  SHOULD  BE  BRIEFED  ALONG  WITH  THE  NORMAL 
APPROACH  BRIEFING  PRIOR  TO  ALL  CLOSELY  SPACED  SIMULTANEOUS  ILS 
APPROACHES. 
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BREAKOUT  PROCEDURE 

OPEN  BOOK  TEST 

1.  When  accomplishing  a  hand  flown  breakout  the  PF  should: 

a.  Wait  for  the  PNF  to  setup  the  flight  directors  before  taking  any  action. 

b.  Immediately  turn  and  point  the  aircraft  (climb  or  descent)  as  directed  by  the  controller. 

c.  Leave  the  aircraft  coupled  to  the  Autopilot. 

2.  The  PNF  needs  to  turn  off: 

a.  Only  his  Flight  Director. 

b.  Only  the  flight  director  of  the  PF. 

c.  Both  flight  directors 

3.  If  the  approach  is  being  made  on  the  autopilot: 

a.  A  breakout  should  be  conducted  on  the  autopilot. 

b.  A  breakout  can  be  done  by  using  the  autopilot  or  by  handflying  the  breakout. 

c.  All  breakouts  must  be  handflown. 

4.  The  PNF  should  turn  on  the  flight  director  of  the  PF: 

a.  Immediately. 

b.  No  urgency,  the  PF  is  experienced  and  has  no  need  for  a  flight  director. 

c.  As  soon  as  the  his  flight  director  matches  the  desired  aircraft  flight  path. 

5.  ( Answer  only  if  aircraft  has  autothrottles)  During  hand  flown  breakouts  the  PF  should: 

a.  Always  keep  the  autothrottles  engaged. 

b.  Consider  disconnecting  the  autothrottles  and  applying  manual  inputs. 

c.  Never  use  autothrottles. 
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l.-B 


2.  -  C 

3. -C 

4.  -  C 

5.  -  B 
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ATIS  A’ 

THIS  IS  CHARLIE  INFORMATION  ALPHA. 

WEATHER  MEASURED  200  OVERCAST,  VISIBILITY  ONE  HALF,  FOG, 
TEMPERATURE  59,  DEW  POINT  54,  WIND  CALM,  ALTIMETER  29.92. 
SIMULTANEOUS  CLOSE  PARALLEL  ILS  RUNWAYS  18L  AND  18R 
APPROACHES  IN  USE. 

ADVISE  YOU  HAVE  ALPHA 


Automatic  Terminal  Information  System  (ATIS)  Card 
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Appendix  F 

Approach  Plates  and  Information  Page 


(INFORMATION  PAGE) 

-  U'Q  J  CHARLIE,  USA 


_ CHARLIE  INTL 

SIMULTANEOUS  CLOSE  PARALLEL  APPROACHES 

18R  18L 


36L  36R 

Runway  Centerline  Separation 
Charlie  International 


The  ATIS  broadcast  will  advise  pilots  when  simultaneous  close  parallel  ILS  approaches  are 
in  progress.  If  unable  to  meet  the  requirements  below,  notify  ATC  immediately. 

Before  initiating  a  simultaneous  close  parallel  ILS  approach,  this  information  bulletin  must 
be  read  and  the  following  requirements  met. 

1.  View  the  FAA  video,  RDU  Precision  Runway  Monitor:  A  Pilot's  Approach. 

2.  All  ATC  directed  breakouts  (a  vector  off  the  ILS  prior  to  the  D/H)  are 
to  be  HANDFLOWN,  unless  company  procedures  and/or  aircraft  flight 
manual  dictate  otherwise. 


BREAKOUT  DESCENTS: 

Pilots  should  be  aware  that  the  controller  may  give  a  descending  breakout.  In  no  case  will  a 
descent  be  given  below  the  MINIMUM  VECTORING  ALTITUDE  (MV A).  The  altitude 
below  which  no  descending  breakouts  will  be  given  by  ATC  is  noted  on  the  approach  plate. 


CONTINUED  ON  OTHER  SIDE 


Pilot  Briefing  Checklist: 

1)  Breakout  briefing,  hand  flown  breakout 

2)  Check  altitude  below  which  no 
descending.breakouts  will  be  given. 

3)  Respond  immediately  to  breakout 
instructions. 
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11-OA 

\ _ y 


CHARLIE,  USA 

_ _ _ _ CHARLIE  INTL 

PHRASEOLOGY: 

Every  effort  will  be  made  by  the  tower  and  monitor  controllers  to  return  aircraft  to  the 
localizer  that  have  overshot  or  strayed  from  the  final  approach  course.  When  an  aircraft 
penetrates  the  NO  TRANSGRESSION  ZONE  (NTZ),  it  is  mandatory  for  the  controller  to 
breakout  aircraft  on  the  adjacent  approach  course  that  might  be  in  danger.  The  phraseology 
for  the  breakout  will  be  as  follows: 

TRAFFIC  ALERT  (aircraft  callsign)  turn  (left/right)  IMMEDIATELY  heading 
(degrees)  Climb/descend  and  maintain  (altitude) . 

Example: 


TRAFFIC  ALERT,  ALL  AMERICA  123,  TURN  RIGHT  IMMEDIATELY  HEADING 
TWO  SEVEN  ZERO,  CLIMB  AND  MAINTAIN  THREE  THOUSAND. 


BLUNDERING 

AIRCRAFT 


The  pilots  receiving  this  message  will  not  be  forewarned  by  hearing  the  transmissions 
to  the  blundering  aircraft,  because  each  approach  course  is  being  monitored  by  a 
separate  frequency.  When  a  pilot  is  told  to  breakoff  the  approach,  the  pilot  must 
assume  the  worst:  that  an  aircraft  from  the  adjacent  localizer  is  heading  his  way  and 
that  the  breakout  must  be  started  immediately  and  continued  as  rapidly  as  safety 
allows.  An  immediate  pilot  response  is  expected  and  required  to  avoid  a 
possible  collision. 
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1 1 1 -1 A 


TWGESSEN  CHARTS 


16  OCT  95 


ATIS  Arrival 

XXX.X 

CHARLIE  Approach  (R) 

XXX.X 

CHARLIE  Tower  Rwys 

18R-36L 

127.2  , 

18L-36R 

126.8 

Ground  121-9 

2000' 


CHARLIE,  USA 

CHARLIE  INTL 

ILS  Rwy  18R 

(CLOSE  PARALLEL) 

,  LOC  108.5 

)  (NOTE:  NO  IDENT) 

MSA 

HANSL  OM  Apt.  Elev  500' 


11-0  MUST  BE  READ  PRIOR  TO  FLYING  THIS  CLOSE  PARALLEL  APPROACH 


Simultaneous  CLOSE  PARALLEL 
approaches  authorized  with  ILS  Rwy  18L. 
RADAR  and  GLIDE  SLOPE  REQUIRED. 


No  descending  breakouts  will  be 
given  below  1800* 


"  ILS  — 

180°  108.5 

(NOTE:  NO  IDENT) 


_  MM 

G$  741’  '(24V) 


_HANSL 
GS  2143'  (1643') 


TCH  55' 

TDZE  500' _ 44 _ W - 


APT.  500'  o  06  50 

MISSED  APPROACH:  Climb  to  1000’  then  climbing  right  turn  to  50001  via  heading  205°  and  expect  radar  vectors. 


STRAIGHT  -  IN  -  LANDING  RWY  18R  CIRCLE  -  TO  *  LAND 

ILS  I  LOC  (GS  Out) 

DAfHI  700'  (2m  _  I _ I 


Not  Authorized 


TWGESSEN  CHARTS 


16  OCT  95 


a]  CHARLIE,  USA 

CHARLIE  INTL 

QILS  Rwy  18L 

(CLOSE  PARALLEL) 

LOCI  09.9 

(NOTE:  NO  IDENT) 

l 

GRETL  OM  Apt  Elev  500' _ 


To  MUST  BE  READ  PRIOR  TO  FLYING  THIS  CLOSE  PARALLEL  APPROACH _  ""1 


ATIS  Arrival  XXX.X 

CHARLIE  Approach  (R)  XXX.X 
CHARLIE  Tower  Rwys  1 8L-36R  1 26.8 

18R-36L  127.2 


Ground  121.9 


No  descending  breakouts  will 
be  given  below  1800’ 


.Simultaneous  CLOSE  PARALLEL 
approaches  authorized  with  ILS  Rwy  1 8R. 
RADAR  and  GLIDE  SLOPE  REQUIRED. 


MM  _  _ 

GS  741’  '  (241’) 

m m 


GRETL _ 

GS  2142'  (1642') 


TCH  55’ 


TDZE  500' _  4.4 _ - — - - - 

APT.  500'  0  06  50 

MISSED  APPROACH:  Climb  to  1000'  then  climbing  left  turn  to  4000'  via  heading  155°  and  expect  radar  vectors. 


STRAIGHT  -  IN  -  LANDING  RWY  18L 
ILS 

DAfH)  750'  (250-) _ 


LOC  (GS  Out) 


CIRCLE -TO -LAND 


|  RVR  1 8  or  1  /2 


RVR  24  or  1/2 


40  or  3/4 


Not  Authorized 


Not  Authorized 


Appendix  G 

Blunder  Distribution  Results 


TABLE  G-l.  SUMMARY  OF  TEST  CONDITION  GOALS  AND  ACTUAL  RESULTS. 


TEST  CONDITION 

Goal  (%) 

Attained  (%) 

Real-Time 

BRM 

ASAT 

Blundering  Aircraft  Communications  Status 

No  Communications 

80.0  i 

79.6 

100.0* 

Communications 

20.0 

20.4 

0.0 

Blundering  Aircraft  Flight  Path 

Maintain  Altitude 

50.0 

53.0 

50.0 

50.0 

Descend 

50.0 

47.0 

50.0 

50.0 

Blundering  Aircraft  Start  Course 

18L 

50.0 

47.7 

50.0  | 

50.0 

18R 

50.0 

52.3 

50.0 

50.0 

Blunder  Distribution  along  Final  Approach  Course 

1-3  nm 

18.0 

16.1 

18.2 

28.6 

3-5  nm 

18.0 

16.8 

18.2 

28.2 

5-7  nm 

18.0 

20.1 

18.2 

12.1 

7-9  nm 

18.0 

16.7 

18.2 

12.6 

9-12  nm 

28.0 

30.3 

27.2 

18.7 

Aircraft  Type  (Involved  in  Blunders) 

Jet 

65.0 

65.1 

68.0 

65.0** 

Heavy  Jet 

30.0 

29.9 

25.8 

30.0** 

General  Aviation 

5.0 

5.0 

6.2 

5.0** 

Aircraft  Flight  System  (Involved  in  Blunders) 

Glass  (Digital) 

60.0 

61.5 

58.6 

Conventional  (Analog) 

35.0 

33.5 

35.2 

General  Aviation 

5.0 

5.0 

6.2 

*  The  BRM  and  AS  AT  simulations  only  used  No  Communications  blunders  in  their  TCV  rate 


calculations  in  order  to  simulate  worst-case  situations. 


**  Actual  percentages  were  not  available,  however,  blunder  distributions  were  selected  based  on 
the  percentage  goals. 
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Site  Coordinator  Briefing 


SITE  COORDINATOR  BRIEFING  MATERIALS 


FOR 

FAA  MPAP  TWG  SIMULATION 


October  16  -  27,  1995 
FAA  Technical  Center 
Atlantic  City  International  Airport,  NJ 
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WELCOME 

You  will  be  participating  as  a  site  coordinator  in  the  simulation  of  close  parallel 
simultaneous  ILS  approaches  to  parallel  runways  at  a  fictitious  airport,  Charlie  International. 

The  runways  of  this  fictitious  airport  are  3000'  apart  and  one  of  the  runways  has  a  2  1/2  degree 
offset  with  the  runway  centerline.  This  Site  Coordinator  Instruction  Package  contains 
instructions  about  how  to  conduct  the  simulation,  how  to  brief  the  pilots,  how  to  administer  the 
pilots  and  how  to  train  the  crews. 

As  the  site  coordinator,  your  responsibilities  include  insuring  pilot  contracts  are  complete 
and  correct,  briefing  the  pilots,  administering  the  pilot  training,  providing  pilots  with  ATIS  card 
and  run  number  information,  documenting  the  approach,  administering  questionnaires,  video 
taping  each  approach,  and  having  the  pilots  complete  a  survey.  In  addition,  site  coordinators 
shall  submit  a  brief  report  containing  their  observations  during  the  simulation  and  operational 
issues  that  it  addressed.  You  are  the  best  source  of  information  concerning  the  pilot  and  cockpit 
perspective  of  the  approach  operation  being  simulated.  Please  record  any  and  all  observations 
carefully  and  completely.  At  any  time  during  the  simulation,  if  you  have  a  question,  please  do 
not  hesitate  to  call  a  SRC  representative. 


***NOTICE*** 


There  have  been  changes  made  to  this  document  since  the  last  simulation.  We  marked  most  of 
the  major  changes  with  a  vertical  bar  in  the  margin.  Please  read  over  the  entire  document  to 
refresh  your  memories.  You  might  get  some  questions  from  the  pilots  with  regard  to  going  over 
the  same  material  and  filling  out  a  similar  survey  to  the  one  they  filled  out  in  August.  Just  tell 
them  that  the  simulation  is  different  and  we  want  them  to  do  everything  again  with  the  exception 
of  viewing  the  film  (we  are  giving  them  the  option  of  not  viewing  it  if  they  viewed  it  in  August). 
The  survey  is  especially  important  because  the  pilot  will  be  answering  the  questions  from  a 
different  point  of  view. 
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GENERAL  INSTRUCTIONS 

1 .  it  is  very  important  that  a  professional  atmosphere  be  maintained  throughout  the  simulation, 
especially  during  final  approaches.  When  the  simulator  is  in  motion,  a  sterile  cockpit  is 
required.  Remember,  the  pilots  are  on  video.  A  number  of  people  will  be  reviewing  this 
simulation  and  many  are  especially  concerned  with  the  professionalism  of  the  participants.  It 
is  essential  for  the  simulation  to  be  conducted  as  close  to  real  world  as  possible. 

2.  Please  do  not  use  the  simulator  phone  lines  at  any  time  for  calls  other  than  to  the  Test 
Director  at  the  FAA  Technical  Center  or  to  contact  a  technician  outside  the  simulator. 

3.  Please  insure  pilots  comply  with  headset  requirements  and  they  use  the  boom  mike  if 
supplied.  The  use  of  the  speaker  might  produce  feedback  into  the  video  sound  and  the  boom 
mike  on  some  of  the  simulators  is  the  source  for  the  cockpit  conversation  recording. 

4.  You  will  be  video  taping  each  two  hour  session  during  the  simulation.  It  is  important  to 
insure  that  communications  between  ATC  and  the  pilots  are  captured  on  the  tape,  as  well  as 
crew  interaction  and  cockpit  conversations.  During  the  last  simulation,  several  simulators  did 
not  record  radio  traffic.  Please  check  your  tapes  to  determine  if  your  site  is  recording  ATC. 
Further  instructions  on  video  taping  can  be  found  later  in  this  document. 

5.  Prior  to  each  approach,  stress  call  sign  recognition  with  the  pilots.  Require  each  pilot  to 
verbalize  the  call  sign  without  reference  to  the  index  card  and  then  have  the  pilots  place  the 
card  so  that  each  pilot  can  see  it.  We  know  that  there  will  be  lots  of  stumbles  over  the  call 
sign,  but  by  using  this  procedure  this  problem  can  be  minimized. 

6.  On  each  approach,  complete  the  Simulator  Pre-Release  Checklist.  We  have  put  the  checklist 
and  the  breakout  questionnaire  in  a  plastic  page  protector  located  in  the  back  of  the  3-ring 
notebook. 

7.  The  pilot  flying  should  alternate  with  each  approach.  If  there  are  two  first  officers  in  the 
seats,  it's  OK  if  they  switch  seats,  so  that  the  pilot  flying  will  be  in  the  seat  in  which  he  is 
most  comfortable. 

8.  It  is  very  important  that  each  approach  is  started  at  the  scripted  time  and  the  aircraft  are  flying 
the  assigned  speed  for  the  approach  number.  This  speed  must  be  maintained  to  the  outer 
marker  unless  changed  by  ATC.  You  are  authorized  to  tell  the  pilots  to  speed  up  or  slow 
down  if  the  speed  is  not  in  the  ballpark.  You  are  not  authorized  to  help  in  any  other  way 
during  the  approaches. 

9.  At  the  end  of  each  day,  fax  all  of  the  site  coordinator  log  sheets  for  the  day  toH^I 

Please  write  legibly  and  remember  that  neatness 
counts.  Retain  the  originals  as  they  will  need  to  be  mailed  at  the  end  of  the  simulation.  If  no 
fax  is  available,  include  a  copy  of  each  log  sheet  when  you  mail  the  contracts  and  expense 
reports. 
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10.  You  will  mail  the  original  contracts  and  expense  reports  for  each  pilot  once  or  twice  a  week 

|  via  priority  mail. 

We  can't  start  the  pay  process  until  we  receive  those  papers. 


DAILY  SCHEDULE 


Training: 

1st  Run  of  Day: 
2nd  Run  of  Day: 
3rd  Run  of  Day: 


1100-  1200  EDT 
1200- 1400  EDT 
1430  -  1630  EDT 
1745  -  1945  EDT 


Please  note  that  the  above  times  are  EASTERN  DA  Y LIGHT  TIME.  The  schedule  will  probably 
be  modified  each  day.  The  times  are  listed  here  just  as  a  guide. 


PILOT  ADMINISTRATION 


Pilot  List  And  Schedule 

You  are  given  a  schedule  and  a  list  of  the  pilots  who  were  assigned  to  your  simulator.  We  tried 
to  get  two  captains  and  one  first  officer  for  each  day.  Since  only  approaches  are  being  conducted 
with  no  emergencies,  we  feel  that  the  first  officers  can  fly  from  the  left  seat  in  the  event  we 
couldn't  get  the  required  captains.  Each  pilot  will  fly  two  periods.  If  there  are  two  captains,  one 
of  the  captains  will  relieve  the  captain  for  one  period  and  the  first  officer  for  one  period.  It  is  not 
necessary  for  the  pilot  who  does  not  fly  the  last  period  to  stay  around  after  he  has  completed  all 
the  necessary  paperwork,  including  the  pilot  survey  on  his  last  day.  It  is  also  not  necessary  for  a 
pilot  who  has  completed  the  training  and  doesn't  start  until  the  second  period,  to  come  in  and 
wait  around  to  fly.  You  decide  how  to  schedule  the  pilots. 

tf  someone  doesn't  show  up,  try  to  get  a  replacement  from  your  pilot  list.  Contact^HHHI 
■■■for  more  names.  You  should  ask  the  pilots  who  did  show  up  if  they 
know  of  anyone  who  would  like  to  participate.  As  a  last  resort  if  two  pilots  don't  show,  you  can 
fly  to  save  the  simulator  period,  but  have  the  scheduled  pilot  fly  all  the  approaches. 
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Pilot  Code  Assignment  Sheet 

You  will  assign  a  separate  letter  code  to  each  pilot  at  the  beginning  of  his/her  participation  in  the 
simulation.  There  is  a  sheet  on  which  this  information  should  be  entered.  The  pilot  will  be 
referred  to  by  this  letter  code  on  the  log  sheets. 

Purchased  Labor  Agreement.  Expense  Account  Form  And  Video  Release  Form  (Samples  of 
filled  out  forms  included) 

Purchased  Labor  Agreement:  have  the  pilot  fill  in  the  name,  address  and  telephone 
number  at  the  top,  sign  the  form  after  acknowledgment  and  insert  his  social  security 
number  and  pilot  certificate  number.  You  will  fill  in  the  dates,  number  of  days  pay  and 
the  total  amount.  If,  for  some  reason,  only  two  pilots  show  up,  pay  the  two  pilots  1  1/3 
days  pay  for  that  day. 

1  Day -WBM 

1  1/3  Day  -  HHI 

Expense  Account  Form:  The  pilot  will  fill  in  his  name  and  address  on  the  top  and  sign 
and  date  the  form  on  the  bottom.  IF  A  PILOT  FLIES  ON  DAYS  OF  BOTH  WEEKS,  HE 
HAS  TO  SIGN  TWO  FORMS,  ONE  FOR  EACH  WEEK.  Explain  that  the  name  and 
address  the  pilot  fills  in  will  be  the  one  our  accounting  department  uses,  so  they  should  be 
careful  that  it  is  readable.  You  will  fill  in  the  numbers  on  the  form  as  per  the  enclosed 
sample. 

Video  Release  Form:  Have  the  pilot  print  his  name,  sign  and  date  the  form. 

You  should  find  a  copy  machine  you  can  use  and  make  copies  of  the  contract  and  expense 
account  form  for  you  and  for  the  pilot.  You  will  send  the  originals  back  to  us  by  US 
priority  mail  every  few  days  so  that  we  can  get  the  paying  process  started.  The  copy 
should  be  sent  back  at  the  end  of  the  simulation  with  all  the  other  materials.  Tell  the 
pilots  to  call  the  SRC  Pilot  Group  if  they  do  not  get  a  check  within  5  to  6  weeks  from  the 
end  of  the  simulation. 

This  signing  in  process  can  take  a  few  minutes.  The  first  day  there  will  be  training,  so  don't 
waste  time.  One  hour  is  calling  it  close  so  it  is  important  to  get  started  right  away. 
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TRAFFIC  SAMPLES.  TRAFFIC  SAMPLE  SCHEDULE,  ATIS  CARD  AND  PREPARING 

CARDS  FOR  EACH  APPROACH 

Three  two  hour  simulator  sessions  will  be  flown  each  day.  Each  two  hour  session  will  be 
referred  to  as  a  RUN.  RUN  #1  will  be  the  first  two  hour  period  on  the  first  day,  RUN  #2  the 
second  two  hour  period,  etc.,  continuing  through  the  simulation  for  a  total  of  30  RUNS  (10  days 
with  3  runs  each).  The  last  run  of  the  two  weeks  will  be  RUN  #30.  There  are  four  sets  of 
TRAFFIC  SAMPLES,  201, 202, 203  and  204.  There  is  a  TRAFFIC  SAMPLE  SCHEDULE 
which  lists  for  each  two  hour  RUN  the  TRAFFIC  SAMPLE  that  will  be  used.  The  TRAFFIC 
SAMPLES  are  the  scripts  for  the  two  hour  simulations  and  consist  of  a  list  of  approaches.  Each 
approach  is  identified  by  a  number  which  is  referred  to  as  an  INDEX  NUMBER.  In  front  of 
many  of  the  INDEX  NUMBERS  is  a  simulation  site  and  how  the  approach  will  be  flown 
(autopilot,  hand  fly,  raw  data  -GAT  only).  The  individual  INDEX  NUMBERS  that  are  identified 
by  your  simulator  will  be  highlighted  and  are  the  ones  that  your  simulator  will  fly.  Cards  have 
been  prepared  for  each  INDEX  NUMBER  for  your  simulator.  Each  site  will  have  a  clock 
synchronized  with  the  FAA  Technical  Center  that  you  will  use  to  place  the  aircraft  into  the 
problem  according  to  the  time  listed  on  the  TRAFFIC  SAMPLE  INDEX  NUMBER.  It  is  the 
responsibility  of  the  site  coordinator  to  verify  that  the  cards  are  in  the  correct  order  and  are 
correct.  Other  information  included  on  the  TRAFFIC  SAMPLE  includes: 

AIRCRAFT  TYPE  -  this  has  no  relation  to  your  simulator  type  and  is  what  will 
appear  on  the  radar  tag.  You  might  be  in  a  MD88  simulator  and  be  assigned  a  run 
number  that  is  a  B-727.  The  pilots  will  always  fly  the  simulator  as  they  normally 
would  even  though  the  aircraft  type  is  different  than  the  simulator.  The  reason  for 
this  is  to  fool  the  controllers  as  to  which  targets  are  simulators  and  which  targets  are 
being  controlled  by  a  computer  operator.  The  pilots  should  be  aware  that  the 
controller  might  refer,  in  this  case,  to  your  target  as  a  727  when  talking  to  another 
aircraft. 

IAS  -  this  speed  is  to  be  held  to  the  outer  marker  unless  changed  by  ATC. 

CALL  SIGN  -  the  call  sign  will  most  probably  be  different  than  the  call  sign  used  by 
the  pilot's  company.  Have  each  pilot  repeat  the  call  sign  out  loud  before  each 
approach. 

INITIAL  ALTITUDE  -  this  will  be  the  altitude  at  simulator  release  and  the  altitude  of 
localizer  and  glide  slope  intercept. 

RUNWAY  ASSIGNMENT-  this  will  show  the  pilot  which  approach  plate  to  use. 

TRANSPONDER  CODE  -  it  is  VERY  IMPORTANT  to  set  prior  to  each  approach. 
The  transponder  code  is  sent  over  the  data  line  to  Atlantic  City  and  identifies  the 
simulator. 
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HAND  FLOWN,  AUTOPILOT,  RAW  DATA  -  describes  how  the  approach  will  be 
flown.  All  simulators  with  exception  of  the  General  Aviaiton  Trainer  (GAT)  will 
only  use  coupled  autopilot  and  hand  flown  (using  flight  director)  approaches.  Each 
approach  will  be  assigned  one  of  these  methods. 

Before  you  hand  the  card  to  the  pilots,  hold  it  up  in  front  of  the  video  camera  and  repeat  out  loud 
the  index  number.  This  will  facilitate  identifying  the  index  number  when  the  tapes  are  reviewed. 

Sometimes  the  FAA  Tech  Center  will  call  during  the  simulation  and  give  you  a  different  index 
number  to  fly.  You  will  prepare  a  new  card  for  that  index  number,  give  it  to  the  pilots  and 
release  the  simulator  at  the  new  appointed  time.  A  highlighter  has  been  provided  for  highlighting 
your  index  number  lines. 

ATIS  Card:  The  TRAFFIC  SAMPLE  will  list  the  ATIS  (A,  B  or  C)  to  be  used  for  each  two  hour 
run.  The  only  difference  in  the  ATIS  will  be  the  wind.  You  will  have  to  put  the  wind  into  the 
simulator  setup  before  each  run  begins.  You  have  been  supplied  two  cards  for  each  ATIS. 


VIDEO  TAPING  INSTRUCTIONS 


Each  two  hour  session  must  be  video  taped.  These  video  tapes  will  be  used  during  data  analysis, 
and  are  a  very  important  part  of  the  entire  data  collection  process.  Before  each  day  begins,  please 
be  sure  that  the  equipment  is  working  properly.  At  the  end  of  each  day  run  the  tape  a  few 
minutes  to  see  if  it  recorded  OK. 

Please  insure  each  video  tape  and  each  video  tape  cover  is  labeled  with  the  following 
information: 


Site,  Date,  Run  Number  (1,2,3  or  4). 

You  will  be  sent  enough  video  tapes  for  the  entire  simulation.  In  case  you  need  more  tapes,  buy 
them  and  put  it  on  your  expense  account. 

The  purpose  of  the  video  is  to  capture  crew  interactions,  communications  between  the  pilots, 
including  pilot  conversation  between  approaches,  and  communications  between  the  cockpit  and 
ATC.  We  know  that  we  won't  be  able  to  see  individual  instruments,  but  we  will  be  able  to  see 
the  pilots'  arm  and  hand  movements. 

Start  the  recording  at  the  beginning  of  each  two  hour  session  and  stop  at  the  end  of  the  session. 
To  insure  that  you  do  not  run  out  of  tape  in  the  middle  of  a  two  hour  run,  please  record  on 
extended  play  format  (or  the  slowest  speed  available)  and  periodically  check  how  much  tape  is 
left.  Know  what  the  slowest  speed  available  is  and  use  it.  In  Atlanta,  for  example,  the  slowest 
speed  available  will  fill  up  the  tape  in  four  hours.  In  this  case,  the  second  two  hour  period  is 
critical  with  regard  to  running  out  of  tape.  During  the  fourth  hour  the  camera  should  be  turned 


H-7 


SITE  COORDINATOR  INSTRUCTIONS 


off  a  minute  after  the  breakout  turn  is  completed  or  when  the  sim  is  stopped  in  case  there  was  no 
breakout.  Do  not  forget  to  hold  the  index  card  in  front  of  the  camera  to  identify  the  run.  This  is 
important  for  the  people  who  are  reviewing  the  210  hours  of  tape.  If  you  are  told  by  Atlantic 
City  that  the  previous  approach  was  a  midair  (see  following  section),  hold  up  the  TCV  shape  in 
front  of  the  camera  lens  for  about  five  seconds.  This  will  help  the  viewer  find  the  approach  that 
resulted  in  the  midair  collision  when  the  viewer  is  using  the  VCR  fast  forward  mode. 

You  may  wait  until  the  end  to  send  all  the  tapes  to  SRC  or  you  may  send  them  back  at  the  end  of 
each  week.  Mail  them  back  in  the  boxes  they  arrived  in  and  use  the  FEDEX  prepaid  labels  in 
your  packet  of  materials. 

SITE  COORDINATOR  LOG 

A  new  site  coordinator  log  has  been  developed.  The  major  difference  is  that  the  sheet  is  divided 
into  two  sections,  a  section  on  the  left  where  every  approach  will  be  recorded  and  a  larger  section 
on  the  right  where  breakout  information  or  any  other  abnormal  information  will  be  recorded. 

Any  abnormalities  noted  during  the  breakout  should  be  recorded  in  the  Abnormalities  column.  A 
simple  number  code  of  common  mishaps  may  be  found  at  the  bottom  of  the  log  to  make  the 
entering  of  comments  easier,  but  if  you  don't  find  a  proper  code,  write  in  your  comment  in  the 
area  provided. 

Please  fill  in  the  date,  run  number,  traffic  sample  and  wind  condition  next  to  the  designated 
spaces  at  the  top  of  the  log.  On  occasion,  the  system  may  fail  during  a  run  and  be  restarted  with 
the  same  Run  Number.  If  this  occurs,  for  example  in  Run  Number  4,  the  traffic  sample  would 
remain  the  same,  and  the  Run  Number  would  become  4-2.  Always  coordinate  with  the  FAA 
Technical  Center  at  the  beginning  of  each  run  to  verify  the  Run  Number  and  the  start  time  of 
your  first  approach.  Please  use  a  new  log  sheet  at  the  beginning  of  each  two  hour  run. 

The  following  sections  will  describe  the  information  required  for  each  column  on  the  Site 
Coordinator  Log: 

Index  Number  -  Refers  to  the  index  number  on  the  Traffic  Sample  sheets.  It  is  the  sequential 
approach  number. 

Call  Sign  -  Refers  to  the  ID  number  of  the  aircraft. 

Pilot  ID  -  Refers  to  the  letter  code  assigned  to  the  pilot. 

Breakout  Questionnaire 

After  each  approach  when  the  pilot  was  vectored  off  of  the  final  approach  course  prior  to 
the  D/H,  you  are  to  ask  the  pilots  the  questions  on  the  Breakout  Questionnaire  which  is 
located  in  the  plastic  envelope  on  the  reverse  side  of  the  pre-release  checklist.  Record  the 
responses  in  the  appropriate  columns  on  the  Site  Coordinator  Log  using  the  appropriate 
codes.  There  should  be  only  one  set  of  responses  for  each  approach.  Either  get  a 
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SITE  COORDINATOR  INSTRUCTIONS 


consensus  of  both  pilots  or  the  answers  from  the  pilot  flying.  Note:  DO  NOT  ACCEPT 
zeroes  or  half  numbers,  iust  whole  numbers. 


YOUR  MOST  IMPORTANT  DUTY  -  Each  time  there  is  a  mid-air  collision  (technical  criteria 
violation  or  TCV  in  simulation  study  jargon)  during  one  of  the  approaches,  all  aspects  of  the 
approach  are  looked  at  in  detail  by  a  number  of  people  from  System  Resources  Corp.  There  are 
three  video  tapes  (the  cockpit,  the  radar  screen  and  the  controllers)  that  are  scrutinized  for  each 
TCV.  The  logs  of  the  controller  observers  and  the  pilot  observers  are  also  gone  over  in  detail. 

We  try  to  determine  why  the  TCV  happened  and  what  we  can  do  to  prevent  it  from  happening 
again.  After  each  TCV  is  scrutinized  by  SRC  personnel,  then  the  TCV  data  is  presented  to  a 
large  group  consisting  of  people  from  FAA  Headquarters,  FAA  operations,  FAA  Technical 
Center  and  several  government  contractors.  Your  pilot  observer  log  will  be  viewed  and  the  video 
of  your  simulator  pilots  will  be  shown  to  this  large  audience.  The  point  is  that  in  spite  of  all  the 
work  that  you  have  done,  a  couple  of  entries  in  your  simulator  log  is  what  is  going  to  be  looked 
at  in  detail.  In  the  past,  the  site  coordinator  never  knew  when  a  midair  or  TCV  occurred.  We 
thought  that  during  this  simulation  we  will  call  you  when  you  have  been  involved  in  a  TCV.  The 
reason  for  this  is  that  you  might  put  down  more  information  if  you  knew  that  the  breakout 
resulted  in  a  mid-air.  However,  we  do  NOT  want  you  to  tell  the  pilots  about  the  TCV.  We  hope 
that  you  put  down  as  much  information  as  possible  after  each  abnormality,  but  make  an  extra 
effort  to  put  down  as  much  as  you  can  recall  after  a  TCV. 

LIST  OF  CONTACTS 


A  list  of  flight  simulator  contacts  and  coordinators,  simulator  facility  and  cab  phone  numbers, 
test  director  primary  and  backup  numbers  has  been  provided.  Each  simulator  site  has  been 
assigned  a  FAATC  Line.  You  are  to  use  this  phone  number  if  you  need  to  reach  the  FAA 
Technical  Center  test  director  at  any  time  during  the  simulation.  In  the  event  the  number 
assigned  to  your  simulator  site  is  busy,  please  use  the  backup  number  provided  for  your  site. 
Each  day  before  the  simulation  starts  you  should  telephone  this  number  and  establish 
contact  with  the  FAA  Technical  Center. 

PTT  OT  BRTF.FTNG  INFORMATION 

A  Pilot  Briefing  pamphlet  was  sent  to  each  pilot  participating.  We  have  included  a  few  extra 
copies  in  your  materials  in  case  a  pilot  signed  on  too  late  to  receive  one.  We  have  included  a 
copy  of  the  letter  that  accompanied  the  pilot  briefing  packet  so  you  know  what  information  the 
pilots  were  supplied. 
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TRAINING 

The  pilots  will  be  trained  before  their  first  ride  in  the  simulator.  The  training  will  consist  of 
watching  a  video,  reading  a  Pilot  Awareness  Training  Bulletin,  and  taking  a  short  test.  All  the 
pilots  except  the  B-727  and  GAT  pilots  will  also  read  a  Breakout  Procedure  Bulletin  and  take  a 
short  test  on  that  subject. 

Video 

Show  the  pilots  the  video,  RDU,  Precision  Runway  Monitor:  A  Pilot's  Approach.  Make 
sure  that  the  door  to  the  room  is  shut  so  that  they  are  not  disturbed.  The  video  runs  for  12 
minutes.  We  expect  that  most  of  the  pilots  will  have  viewed  the  video  during  the 
previous  simulation.  They  are  not  required  to  see  it  again,  but  may  view  it  if  they  wish. 

It  also  might  be  a  good  idea  for  you  to  review  it  again. 

Pilot  Awareness  Training  Bulletin  and  Procedures  Training  Bulletin  (if  applicable) 

After  the  video,  hand  out  the  approach  plates,  airport  information  page,  Pilot  Awareness 
Bulletin  and  the  Procedures  Training  Bulletin  (if  applicable  -  only  for  glass  cockpits).  Do 
not  hand  out  the  tests  at  this  time,  just  tell  them  that  there  will  be  a  test. 

The  bulletin  type  training  is  similar  to  a  pilot  going  to  work  and  finding  a  training  bulletin 
in  his  mail  box  together  with  a  test.  The  pilot  would  read  the  bulletin,  take  a  short  test 
and  hand  it  in  to  the  chief  pilot  so  that  the  pilot’s  training  records  could  reflect  that  he  had 
received  training.  You  are  not  an  instructor,  but  you  can  answer  questions  if  asked.  This 
would  be  similar  to  a  pilot  with  a  question  about  one  of  the  training  bulletins  going  into 
see  one  of  the  check  pilots  to  get  an  answer. 

The  pilots  will  grade  their  own  tests.  The  answers  are  on  the  back  of  the  tests.  Don't 
hand  out  the  tests  until  they  read  the  bulletins  and  don't  tell  them  they  are  self-graded 
until  you  hand  out  the  tests.  We  don't  need  the  tests  back,  but  please  check  to  insure  that 
they  got  the  correct  answers.  We  found  in  one  of  our  previous  simulations  that  a 
significant  number  of  pilots  put  down  the  wrong  answer  because  the  question  was  not 
covered  in  the  training.  Note  if  more  than  one  pilot  gets  one  of  the  questions  wrong. 

PILOT  SURVEY 

On  a  pilot's  last  day,  he  will  fill  out  a  survey.  Please  stress  to  him  that  it  is  very  important 
to  put  down  all  the  comments  that  come  to  mind.  Make  sure  that  the  pilot  leaving  early 
finishes  the  survey  before  he  leaves.  DON'T  LET  HIM  LEAVE  WITHOUT 
COMPLETING  THE  SURVEY.  The  survey  is  a  very  important  part  of  the  simulation 
and  we  have  used  the  pilots'  comments  to  get  procedures  changed  in  the  past.  If  we  think 
that  something  is  needed  for  safety  and  it  costs  money  to  implement,  it  is  much  easier  to 
get  approval  if  we  have  support  from  the  pilots  taking  part  in  the  simulation.  Many  of  the 
procedures  in  this  simulation  are  a  result  of  pilots’  comments  in  the  past  and  were  not 
achieved  easily. 
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FINAL  REPORT 

We  need  a  final  report  from  you  to  help  us  in  this  simulation  and  in  future  simulations. 
We  would  like  a  summary  of  your  site,  what  went  wrong,  what  went  right,  what  we 
should  have  done,  what  we  did  wrong,  what  you  observed  as  the  problems  with  the  close 
parallel  approaches,  what  you  would  recommend  to  make  them  safer,  what  we  could  do 
to  make  the  simulations  more  realistic,  etc.  You  will  be  on  the  front  lines  observing  first 
hand,  so  your  comments  will  be  especially  pertinent.  Some  of  you  gave  very  detailed 
reports  in  August  and  we  truly  appreciate  the  time  and  effort  you  made.  We  hope  that 
every  site  coordinator  gives  us  a  detailed  report  after  this  simulation.  This  time  you  will 
know  that  your  simulator  was  involved  in  a  midair  collision  and  can  probably  give  us  a 
better  insight  as  to  why.  You  might  be  able  to  give  us  ideas  about  better  training,  better 
pilot  technique,  better  controller  controlling,  etc.  You  might  conclude  that  3000'  is  too 
close.  You  might  conclude  that  all  is  OK  and  we  should  implement  this  tomorrow.  You 
might  be  able  to  pick  up  something  that  we  didn't  observe  before  (a  site  coordinator  in 
August  picked  up  a  safety  item  we  never  knew  existed).  Any  and  all  of  your  comments 
are  appreciated.  You  never  can  tell  us  too  much. 

THE  END 

At  the  end  of  the  simulation,  pack  up  all  the  materials  and  send  them  back  to[|^B| 

we 

should  already  have  a  signed  copy  of  your  Purchased  Labor  Agreement.  If  you  have 
expenses,  put  all  your  receipts  in  an  envelopealon^id^^etailed  accounting  of  your 
expenses,  SIGN  THREE  expense  sheets|fl|HHHHand  send  it  all  back  to  SRC. 
We  will  fill  in  the  expense  reports  for  you  with  your  receipt  information.  All  telephone 
calls  must  be  identified  with  date,  who  called,  why  and  from  where  to  where.  We  will  do 
our  best  to  get  your  pay  as  soon  as  we  can.  We  realize  that  some  of  you  have  put  out  a 
considerable  amount  of  money  and  need  to  be  reimbursed  without  delay. 

Those  of  us  at  SRC  and  our  customers  at  the  FAA  join  to  thank  you  for  your 
participation.  You  have  made  a  difference  in  something  that  might  or  might  not 
happen  in  the  future  of  aviation.  Not  many  pilots  get  that  chance.  Close  parallel 
approaches  might  not  happen  at  3000'  runway  separation,  but  if  they  do  or  don't 
you  have  had  an  input  into  the  final  decision. 
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18R 


"PA31" 


IP-TIME:  1:21:58 


"N84986" 


HDG 

:  200° 

ALT:  4500' 

IAS:  1 26  kts 

LOC: 

108.5 

TOWER:  1  27.2 

SQUAWK:  2 1  36 

Raw  Data 

SITE: 

GAT 

SAMPLE:  201 

INDEX#:  95 

Approach  Information  Index  Card 
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Controller  Questionnaires 


CONTROLLER  QUESTIONNAIRE 

October  16-27,  1995 

PARTICIPANT  CODE  _  DATE  _ 

Please  complete  the  following  questionnaire  based  upon  your  experiences  throughout  the  day 

(practice  runs  excluded).  Please  refer  to  the  list  of  definitions  on  the  back  of  this  questionnaire, 

if  necessary.  Should  you  have  any  questions,  ask  the  SRC  representative. 

1 .  Rate  the  level  of  stress  experienced  to  resolve  blunders. 

1  2  3 

minimal  moderate 

2.  Rate  the  level  of  stress  experienced  during  the  entire  day 

12  3  4 

minimal  moderate 

3.  Rate  the  mental  effort  required  to  detect  blunders. 

1  2  3  4  5 

minimal  moderate  intense 


4  5 

intense 

5 

intense 


4.  Rate  the  mental  effort  required  to  resolve  blunders. 


minimal 


moderate 


5 

intense 


5.  Rate  the  level  of  mental  effort  required  during  the  entire  day. 


1 

2 

3 

4 

5 

minimal 

moderate 

intense 

6.  Rate  the  level  of  activity  required  during  the  entire  day. 


1 

2 

3 

4 

5 

minimal 

moderate 

intense 
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Please  comment  on  any  occurrences/issues  throughout  the  day,  for  example  problems 
with  visuals,  communications,  aircraft  performance,  long  delays,  pilot  response,  and  other 
outside  factors  that  may  have  affected  your  work  environment. 


Thank  you  for  your  cooperation  with  completing  this  questionnaire. 


DEFINITIONS 

The  following  list  provides  you  with  a  guideline  for  each  of  the  measures  used  in 
the  Controller  Questionnaire. 

STRESS:  (anxiety,  frustration,  pressure) 

•  Experienced  to  re-establish  separation  during  a  blunder 

•  Experienced  when  working  with  the  PRM  system 

•  Experienced  in  remaining  alert 

MENTAL  EFFORT:  (concentration,  attention,  decision-making) 

•  Experienced  in  trying  to  maintain  vigilance 

•  Attention  involved  in  detecting  and  resolving  blunders 

•  Experienced  because  of  the  equipment 
ACTIVITY:  (frequency  of  actions,  time  spent  on  tasks) 

•  Workload 

•  Number  of  blunders 

•  Traffic  density 
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POST-SIMULATION  CONTROLLER  QUESTIONNAIRE 


October  16-27,  1995 


NOTE:  All  responses  provided  from  the  following  questionnaire  will  be  reported  as  an 

aggregate.  Individual  responses  will  not  be  reported.  To  ensure  complete  anonymity,  please  do 
not  write  your  name  or  controller  letter  on  questionnaire.  Thank  you. 
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POST-SIMULATION  CONTROLLER  QUESTIONNAIRE 

October  16-27,  1995 

Please  complete  the  following  questionnaire  based  upon  your  experiences  throughout  the 
simulation.  If  you  need  additional  space,  use  the  back  of  the  sheet.  Should  you  have  any 
questions,  ask  the  SRC  representative. 


1.  Dual  simultaneous  ILS  approaches  to  runways  spaced  3000  ft  apart  with  one 
localizer  offset  by  2.5  degrees  can  be  safely  conducted  as  simulated.  Explain. 

12  3 

Yes  No  Don't  Know 


lb.  What  improvements  could  be  incorporated  into  the  procedures,  if  any? 


2.  Assess  the  communications  workload  for  NTZ  entries  and  breakouts  relative  to 
current  operations.  Please  explain. 

1  2  3  4  5 

Much  less  than  Less  than  Equal  to  Greater  than  Much  greater  than 
current  operations  current  operations 


3.  What  specific  control  strategy,  if  any,  did  you  develop  for  the  simulated  approach 
operation  (e.g.,  inter-controller  coordination,  display  scanning  techniques)? 


3b.  Would  these  strategies  be  specific  to  this  operation? 


4.  Assess  the  adequacy  of  the  briefing  information  and  training  aids  (e.g.,  video 
training  booklet)  you  were  given  prior  to  working  the  monitor  position. 


5.  Rate  the  realism  of  the  traffic  (e.g.,  aircraft  types,  density).  Please  explain. 


1  2  3  4  5 

Low  Moderate  High 


6.  Were  there  any  cues  that  you  used  to  anticipate  the  occurrence  of  blunders?  Please 
describe. 


12  3 

Yes  No  Don't  Know 
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7.  Please  describe  any  items  in  the  simulation  which  you  believe  were  not  realistic  or 
whose  realism  could  have  been  improved  upon  (include  any  comments  about 
equipment,  displays,  communication,  etc.). 


Thank  you  for  participating  in  this  simulation! 
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Flight  Simulator  Pilot  Questionnaires 


PILOT  BREAK-OUT  QUESTIONNAIRE 

(WRITE  IN  ANSWERS  IN  THE  "Q"  COLUMNS  ON  COORDINATORS’  LOG) 

October  16  -  27,  1995 

Ql.  WAS  THE  BREAK-OUT  INSTRUCTION  COMMUNICATED  CLEARLY  AND  CONCISELY? 

(E.G.,  RATE  OF  SPEECH,  CLARITY,  VOLUME,  ETC.) 

PUT  ”Y"  OR  "N"  IN  BLOCK  "Ql" 

IF  NO,  STATE  REASON  IN  THE  COMMENTS  SECTION. 


Q2.  IN  WHAT  ORDER  WAS  THE  BREAK-OUT  INSTRUCTION  GIVEN  AND  HOW  MANY 

TRANSMISSIONS  DID  IT  TAKE  FOR  THE  INITIAL  INSTRUCTIONS? 

1  -  HEADING,  ALTITUDE  (IN  ONE  TRANSMISSION) 

2  -  ALTITUDE,  HEADING  (IN  ONE  TRANSMISSION) 

3  -  HEADING,  ALTITUDE  (IN  TWO  SEPARATE  TRANSMISSIONS) 

4  -  ALTITUDE,  HEADING  (IN  TWO  SEPARATE  TRANSMISSIONS) 

5 -OTHER 

IF  5  -  OTHER,  PLEASE  DESCRIBE  IN  THE  COMMENTS  SECTION. 


Q3.  DID  THE  CONTROLLER  GIVE  YOU  MORE  THAN  ONE  TURN? 

1  -  ONE  TURN.  2  -  TWO  OR  MORE  INCREMENT  TURNS 


Q4.  GIVEN  THE  CONTROLLER  INSTRUCTION,  AIRCRAFT  CONFIGURATION,  AND  FLIGHT 
REGIME,  RATE  THE  DIFFICULTY  OF  THE  BREAK-OUT  MANEUVER. 

1  2  3  4  5 

NOT  DIFFICULT  VERY  DIFFICULT 

IF  DIFFICULT  WRITE  REASONS  IN  COMMENTS  SECTION 


Q5.  DID  THE  BREAKOUT  INSTRUCTIONS  BEGIN  WITH  "TRAFFIC  ALERT"  ? 

PUT  "Y"  OR  "N"  IN  BLOCK  "Q5" 


Q6.  WHAT,  IF  ANY,  ADDITIONAL  COMMENTS  DO  YOU  HAVE? 

WRITE  THEM  IN  THE  COMMENTS  SECTION 
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FLIGHT  CREW  OPINION  SURVEY 

October  16  -  27,  1995 


DATE: _ _ _  PILOT  CODE: _ 

TOTAL  FLIGHT  HOURS _ 

HOURS  IN  TYPE _ 

LAST  TIME  IN  A  SIMULATOR  (mo/yr) _ _ 

SITE:  AVIA  GAT  MD90  NASA  OK  CITY 

Please  circle  the  appropriate  response  for  all  of  the  following  questions,  and  provide  feedback  on  the 
lines  provided.  If  necessary,  continue  feedback  on  the  back  of  the  page.  If  you  have  any  questions, 
consult  your  Site  Coordinator. 


1.  Based  on  the  information  presented  on  your  approach  plates  and  airport  information  page,  in 
combination  with  the  simulated  weather  conditions,  how  would  you  choose  to  fly  close  parallel 
approaches? 


(1)  Coupled  Autopilot  (2)  Flight  Director  (3)  Raw  Data 


Comments: 


2.  The  Airport  Information  Page  increased  my  awareness  of  possible  traffic  in  close  proximity  on 


the  adjacent  approach. 

1 

2 

3 

4 

5 

Strongly  Disagree 

Neutral 

Strongly  Agree 

Comments: 

J-2 


3.  More  crew  coordination  is  required  for  simultaneous  close  parallel  approaches  than  for  normal 
ILS  approaches. 


1 

2 

3 

4 

5 

Strongly  Disagree 

Neutral 

Strongly  Agree 

Comments: 

4.  The  Airport  Information  Page  increased  my  awareness  of  simultaneous  close  parallel  approach 


procedures. 

1 

2 

3 

4 

5 

Strongly  Disagree 

Neutral 

Strongly  Agree 

Comments: 

5.  The  new  ATC  phraseology  ("Traffic  Alert")  coupled  with  the  word  "immediately"  encouraged 
me  to  respond  more  quickly  to  the  breakout  maneuver  than  I  would  have  if  only  the  word 


"immediately"  were  used. 

1  2 

3 

4 

5 

Strongly  Disagree 

Neutral 

Strongly  Agree 

Comments: 

6.  The  video  increased  my  awareness  of  simultaneous  close  parallel  approach  operations. 


1 

2 

3 

4 

5 

Strongly  Disagree 

Neutral 

Strongly  Agree 

Comments: 
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7.  The  training  bulletins  ( Pilot  Awareness  and  Procedure-if  applicable)  increased  my 
understanding  of  what  is  expected  of  me  during  simultaneous  close  parallel  approaches. 


1 

2 

3 

4 

5 

Strongly  Disagree 

Neutral 

Strongly  Agree 

Comments: 

- 

8.  What  additional  items  would  you  like  to  see  on  the  close  parallel  simultaneous  approach  plates? 


9.  What  additional  information  would  you  like  to  see  on  the  Airport  Information  Page? 


10.  The  training  bulletins  helped  me  to  execute  the  ATC-directed  breakouts. 


1 

2 

3 

4 

5 

Strongly  Disagree 

Neutral 

Strongly  Agree 

Comments: 

1 1 .  What  additional  information  or  training  would  you  like  to  have  for  simultaneous  close  parallel 
approach  procedures? 
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12.  In  your  estimation,  based  on  your  experience,  what  percentage  of  ILS  approaches  are  flown  on 
autopilot  to  Category  I  minima  during  IMC  in  normal  line  operations  for: 

(a)  Glass  Cockpit: 


(1)  0-25 

(2) 

26-50 

(3) 

51-75 

(4) 

76  - 100 

(5) 

Don't  Know 

(b)  Conventional: 

(1)  0-25 

(2) 

26-50 

(3) 

51-75 

(4) 

76  - 100 

(5) 

Don’t  Know 

Comments: 

13.  The  no  descending  breakout  altitude  shown  on  the  approach  plate  helped  me  better  prepare  for 
the  possible  breakout  maneuver. 


1 

2 

3 

4 

5 

Strongly  Disagree 

Neutral 

Strongly  Agree 

Comments: 
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14.  Please  list  the  importance  (from  1  to  10)  of  each  of  the  following  items  to  a  pilot  flying  a  close 
parallel  simultaneous  approach.  l=Not  Important,  10=Very  Important 


Item 

Rating  1  to  10 

Video 

Airport  Information  Page 

Separate  Approach  Plate  for  Close  Parallel  Simultaneous  Approaches 

Notes  in  box  on  Approach  Plate 

Pilot  Awareness  Bulletin 

Breakout  Procedure  Bulletin  (MD  -  90  /  B  -  747  -  400  ONLY) 

The  use  of  words  "Traffic  Alert"  in  the  breakout  instruction 

No  descending  breakout  altitude  printed  on  approach  plate 

■■■ 

15.  Additional  comments  on  the  simulation  or  on  the  tested  procedure. 

Thank  you  for  completing  this  survey  and  participating  in  the  simulation.  If  you  would  like  to  have  a 
copy  of  the  final  test  report  sent  to  you,  please  complete  the  form  on  the  next  page. 
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Observer  Logs 


TECHNICAL  OBSERVER  LOG 
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Additional  Comments; 
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Appendix  L 

Aircraft  Performance  Parameters 

In  addition  to  the  pilot/aircraft  response  times  in  Section  1 1. 6.4.2,  maximum  aircraft  performance 
parameters  were  also  extracted  from  the  data  collected  at  the  flight  simulator  sites.  The 
distribution  of  maximum  bank  angles  achieved  during  the  evasion  maneuver  is  shown  in  Figure 
L-l.  It  should  be  noted  that  Figure  L-l  does  not  indicate  how  long  these  bank  angles  were 
maintained.  The  data  indicated  that  they  were  maintained  anywhere  from  momentarily  to  several 
seconds. 


Figure  L-l .  Maximum  Bank  Angle  During  Evasion  Maneuver 

Other  maximum  aircraft  performance  parameters  that  were  extracted  included  the  average 
maximum  roll  rate  (Figure  L-2)  and  average  maximum  rate  of  climb  (Figure  L-3).  “Average 
maximum”  is  the  highest  five-second  average  of  roll  rate  or  rate  of  climb  during  the  evasion 
maneuver.  The  distributions  include  a  sampling  of  at  least  24  data  points  from  each  jet  simulator 
site  and  all  15  data  points  from  the  GAT. 


Figure  L-2.  Average  Maximum  Roll  Rates  During  Evasion  Maneuver 
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Figure  L-3.  Average  Maximum  Rates  of  Climb  During  Evasion  Maneuver 
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Appendix  M 

Missed  Approach  Evaluation 


In  January  1996,  FAA  Standards  Development  Branch  (AFS-450)  conducted  an  evaluation  of  the 
missed  approach  to  ELS  Runway  22R  at  John  F.  Kennedy  Airport  (JFK),  New  York.  The 
localizer  course  to  JFK  Runway  22R  has  a  2.5-degree  offset  from  the  runway  centerline.  The 
localizer  offset  creates  a  slightly  converging  approach  and  missed  approach  situation  relative  to 
Runway  22L,  which  is  3000  feet  (ft)  from  Runway  22R.  The  objective  of  the  evaluation  was  to 
acquire  missed  approach  track  data  to  determine  the  degree  of  safety  and  operational 
acceptability  of  conducting  missed  approaches  with  a  3-degree  localizer  offset  to  closely  spaced 
parallel  runways. 

The  evaluation  utilized  the  FAA/NASA  B747-400  flight  simulator  located  at  NASA  Ames,  San 
Jose,  California.  The  flight  crews  flew  missed  approaches  and  landings  as  prescribed  by  the  test 
director.  The  aircraft  weight  was  set  at  650,000  pounds  to  emulate  the  maximum  landing  weight 
and  a  decision  height  of  250  ft  was  used.  Several  flight  parameters  (x,  y,  and  z,  etc.)  were 
recorded  to  document  the  crew/aircraft  track  performance.  Prior  to  the  simulator  test  session, 
each  crew  was  briefed  on  the  purpose  of  the  test  and  after  the  simulator  session  a  questionnaire 
was  administered  to  each  pilot. 

The  results  of  the  evaluation  were  based  on  ten  qualified  B747-400  crews  which  flew  a  total  of 
78  missed  approaches.  The  results  are  summarized  as  follows: 

•  Eight  flight  tracks  of  the  78  total  missed  approaches  entered  the  NTZ. 

•  The  mean  lateral  overshoot  of  the  centerline  was  334.3  ft  with  a  standard  deviation  of 
110.4  ft. 

•  The  maximum  lateral  penetration  of  the  NTZ  was  205.3  ft. 

•  Flight  track  rotational  analysis  revealed  that  the  NTZ  penetrations  during  the  missed 
approach  were  due  to  the  3-degree  localizer  offset. 

•  The  pilot  questionnaire  revealed  that  the  crews  found  the  approaches  and  missed  approaches 
with  a  3-degree  localizer  offset  and  a  250  ft  decision  height  very  acceptable.  The  pilots  made 
the  following  suggestions:  crosswinds  should  be  limited  to  10  knots;  autoland  operations 
should  not  be  allowed;  and  Air  Traffic  Control  (ATC)  should  monitor  the  missed  approach  to 
give  corrective  action  as  needed. 

From  the  test  results,  it  can  be  concluded  that  if  a  near  gaussion  distribution  is  assumed  for  the 
missed  approach  tracks,  approximately  seven  percent  of  the  missed  approaches  would  enter  the 
NTZ.  Therefore,  ATC  monitoring  of  the  missed  approach  should  be  required  to  insure  adequate 
separation  in  the  event  of  a  simultaneous  missed  approach  on  parallel  runways  spaced  3000  ft 
apart  with  a  localizer  offset  up  to  3  degrees. 
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Additional  evaluation  was  accomplished  by  AFS-450  with  the  Airspace  Simulation  and  Analysis 
for  TERPS  (ASAT)  computer  model.  Computer  software  was  developed  to  characterize  the  JFK 
airport  which  included  the  approaches  and  missed  approaches  for  Runway  22L  and  22R. 

Runway  22R  has  a  2.5-degree  localizer  offset.  In  this  evaluation,  approaches  and  missed 
approaches  were  performed  with  a  50  percent  mix  of  B727  and  B747  aircraft.  The  missed 
approach  procedure  was  performed  as  per  the  Jeppesen  charts,  as  follows: 

Runway  22L:  Climb  to  500  feet,  then  climbing  left  turn  to  3000  ft  via  JFK  R-190  to 
CHANT  Int/19  DME  and  hold. 

Runway  22R:  Climb  to  4000  ft  direct  COL  VOR/DME  and  hold. 

The  operational  procedure  of  the  two  aircraft  types  was  as  follows: 


B727 

B747 

Approach  (Kts) 

130-138 

135-140 

Climb  (Kts) 

185-200 

200-230 

Climb  at  ROC  (Fpm) 

1500 

1500 

Test  parameters  were  varied  from  run  to  ran  based  on  pre-defined  distributions  and/or  pre¬ 
defined  ranges.  The  following  parameters  were  varied: 

•  Aircraft  type  on  each  runway 

•  Missed  approach  distance  from  runway  threshold 

•  Lateral  and  vertical  distance  from  ILS  centerline 

•  Height  loss  induced  by  aircraft  load  factor 

•  Altitude  at  which  the  aircraft  going  around  on  Runway  22L  starts  turning  to 
heading 

The  ASAT  system  performed  10,000  Monte  Carlo  type  simulations  of  the  approaches  and  missed 
approaches  to  Runways  22L  and  22R  at  JFK  airport.  The  specified  operational  parameters  were 
varied  within  normal  ranges  and  no  blunder  scenarios  were  tested.  The  results  of  this  evaluation 
are  as  follows: 

•  During  the  missed  approach,  the  closest  point  between  the  center  of  the  two 
aircraft  targets  was  approximately  1950  ft. 

•  The  maximum  lateral  penetration  of  the  NTZ  was  approximately  200  ft  for 
Runway  22R  with  a  2.5-degree  localizer  offset. 
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The  results  of  the  AS  AT  evaluation,  like  the  B747-400  test,  revealed  that  penetration  of  the  NTZ 
between  the  runways  can  be  expected.  ATC  monitoring  of  the  missed  approach  should  be 
required  to  insure  separation  in  the  event  of  a  simultaneous  missed  approach. 
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